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Abstract 

This  study  used  the  Poisson  regression  technique,  bootstrap  estimates,  and  the 
Kaplan-Meier  estimates  for  survivor  curves  to  determine  the  impact  of  Programmed 
Depot  Maintenance  (PDM)  on  weapon  system  availability.  More  specifically,  these 
techniques  estimated  the  effect  in  the  number  failures  per  year  due  to  PDM,  but  the 
bootstrap  technique  also  estimated  the  effect  in  the  amount  of  downtime  experienced  by  a 
weapon  system  due  to  PDM.  Although  the  Poisson  regression  model  did  not  pass  the 
rigors  of  statistical  testing,  the  Poisson  regression  suggested  differences  in  the  number  of 
failures  per  year  between  PDM  and  no-PDM  weapon  systems.  The  results  of  the 
bootstrap  estimates  for  the  number  of  failures  per  year  and  amount  of  downtime  per  year 
showed  that  the  no-PDM  weapon  systems  experienced  approximately  1 .3  failures  per  year 
while  remaining  unserviceable  approximately  120  days  per  year,  and  PDM  weapon 
systems  experienced  approximately  .3  failures  per  year  while  remaining  unserviceable 
approximately  30  days  per  year.  PDM  reduced  the  number  of  failures  per  year  and 
drastically  reduced  weapon  system  downtime  per  year.  PDM  increased  weapon  system 
availability  fi’om  approximately  67  percent  to  approximately  92  percent. 
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MEASURING  THE  IMPACT  OF  PDM  FUNDING  SHORTFALLS  ON  WEAPON 


SYSTEM  AVAILABILITY 


I.  Introduction 


All  systems,  whether  in  the  Department  of  the  Defense  (DoD)  weapon  system  inventory 
or  not,  eventually  fail.  Early  in  a  weapon  system’s  life  cycle,  these  failures  are  relatively  minor 
and  can  be  quickly  repaired  at  the  base  level.  However,  as  a  weapon  system  ages,  failures  can 
occur  more  frequently.  The  probability  of  a  major  failure,  which  can  not  be  repaired  at  base  level, 
also  increases.  To  minimize  the  effects  of  age  and  major  failures  of  a  weapon  system,  the  DoD 
developed  Programmed  Depot  Maintenance  (PDM).  “The  Department  of  Defense  Dictionary  of 
Military  and  Associated  Terms  defines  depot  maintenance  as  ‘...maintenance  performed  on 
materiel  requiring  major  overhaul  or  a  complete  rebuild  of  parts,  assemblies,  subassemblies,  and 
end  items,  including  the  manufacture  of  parts,  modifications,  testing,  and  reclamation  as 
required’”  (Joint  Logistic  Commanders,  1988: 14).  When  the  weapon  system  leaves  the  depot,  it 
is  considered,  for  all  intents  and  purposes,  a  brand  new  weapon  system. 

Unfortunately,  PDM  is  a  very  expensive  process  and  “across  the  services  represents  an 
approximately  $10  billion  dollars  a  year  business”  (Joint  Logistic  Commanders,  1988;  11).  “Both 
the  House  Appropriations  Committee  (HAC)  and  the  Senate  Appropriations  Committee  (SAC) 
have  questioned  the  credibility  of  the  depot  maintenance  requirements  as  reflected  in  the 
President’s  Budget  and  have  expressed  this  concern  in  the  FY89  budget  markup”  (Joint  Logistic 
Commanders,  1988:12). 
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The  HAC  and  SAC  believe  that  there  is  a  hierarchy  of  maintenance  functions  and  that 
there  should  be  an  emphasis  to  perform  maintenance  at  the  lowest  level  (Joint  Logistic 
Commanders,  1988:11),  This  philosophy  threatens  the  DoD  budget  for  PDM.  TheDoDis 
concerned  because  the  DoD  believes  PDM  increases  the  weapon  system  availability.  This  study 
will  attempt  to  quantify  the  increase  in  a  weapon  system’s  availability  due  to  PDM.  Chapter  1 
will  provide  the  motivation  for  the  study  and  the  PDM  budget  process,  explain  the  concept  of 
PDM,  provide  a  brief  problem  statement  and  present  the  scope  of  the  data. 

Backaround 

The  current  methodology  for  determining  PDM  funding  within  the  DoD  does  not 
effectively  relate  requirements  to  readiness  (Bachmann,  1994:21).  As  a  result,  the  degradation  of 
readiness  or  sustainability  because  of  shortfalls  in  depot  maintenance  funding  has  not  been 
quantified  (Bachmann,  1994:21).  In  order  to  justify  Congressional  funding  for  PDM,  the  DoD 
needs  to  develop  a  model  with  the  capability  to  link  resource  allocation  to  actual  unit  readiness 
and  then  readiness  directly  to  PDM  funding.  Currently,  no  standardized  model  exists  among  the 
Services.  This  creates  a  major  problem  in  prioritizing  weapon  system  funding  between  Services. 

A  top  down  modeling  effort  can  address  this  shortfall.  More  specifically,  a  model  must  be 
conceived  that  formally  addresses  the  problem  of  linking  unit  readiness  to  PDM  funding. 
Furthermore,  this  model  must  be  robust  and  easily  applied  to  other  weapon  systems  (Byrd,  1994: 
1).  This  study  will  attempt  to  illustrate  the  importance  of  PDM  for  a  test  case  and  develop  a  more 
comprehensive  architecture  for  relating  readiness  to  PDM  funding  shortfalls  that  can  be  applied 
across  all  the  Services. 
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The  DoD  uses  a  variety  of  quantitative  and  qualitative  resource  measures  to  indicate  unit 
readiness,  but  they  primarily  focus  on  Status  Of  Resource  and  Training  Systems  (SORTS)  (C- 
ratings)  and  Mission  Capable  (MC)  rates  of  weapon  systems  (JGLR-DMBA  sub  group  meeting). 
SORTS  measures  the  “month  to  month  readiness  of  personnel,  equipment,  and  training  of 
operational  forces”  (Curtin,  1994: 1).  It  ranges  from  a  fully  mission  capable  status  of  C-1  to  a  not 
mission  capable  status  of  C-4  with  C-5  indicating  unit  deactivation.  This  measure  targets  an 
organization’s  availability  and  capability  to  fight,  but  does  not  capture  the  availability  of  an  entire 
inventory  of  a  weapon  system.  Therefore,  SORTS  can  not  be  used  to  determine  the  allocation  of 
PDM  funding. 

MC  rate  is  defined  as:  The  “number  of  items  in  maintenance  over  a  set  number  of  days” 
(Gebicke,  1995:8).  This  is  a  good  indicator  for  weapon  system  availability,  but  gives  no  insight 
on  variability  of  the  downtime  distribution.  Because  of  the  variety  of  missions  subjected  to  the 
different  weapon  systems,  as  well  as  the  differences  inherent  in  each  weapon  system’s  failure 
distributions,  the  downtime  distributions  are  presumably  different  for  each  weapon  system.  In 
addition,  each  distribution  may  have  a  unique  variance.  Depending  on  the  actual  downtime 
distribution  of  a  weapon  system,  the  variability  of  the  distribution  can  be  quite  large,  and  therefore 
it  can  be  difficult  to  compare  MC  rates  across  different  weapon  systems.  Because  of  these 
difBculties,  the  DoD  should  not  solely  depend  on  MC  rates  for  the  allocation  of  PDM  funding. 

When  dealing  with  readiness,  the  Commanders-in-Chief  focus  on  their  Critical  Items  List 
(CIL)  (The  Readiness  Impacts  of  Depot  Maintenance  UDR,  1994: 10).  The  CIL  consists  of 
essential  weapon  systems,  as  seen  by  the  respective  CINC,  to  conduct  a  war  fighting  operation. 
This  list  is  limited  to  120  end  items  (The  Readiness  Impacts  of  Depot  Maintenance  UDR, 

1994: 10).  The  Joint  Chiefs  of  Staff  then  compile  all  of  these  lists  and  develop  a  composite  list. 
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the  composite  CINC  CIL,  of  150  items.  This  study  will  attempt  to  provide  a  framework  to 
analyze  the  relationship  between  shortfalls  in  PDM  funding  and  equipment  availability  that  can  be 
applied  to  all  items  on  the  CINC  CIL. 

The  DoD  tracks  depot  maintenance  funding  with  depot  maintenance  backlog  dollars  and 
percentage  of  requirements  funded.  Backlog  equates  to  Unfiinded  Deferred  Requirements 
(UDR).  In  the  Air  Force,  each  depot  forecasts  a  total  requirement  for  each  weapon  system  for 
the  next  fiscal  year.  Other  services  rely  on  inputs  from  T5rpe  Command  or  Combat  Command. 
Unfortunately,  the  total  requirement  can  not  be  serviced  within  that  fiscal  year  due  to  operational 
commitments,  capability  requirements,  and  other  unexecutable  requirements.  The  total 
requirement  minus  the  prior  operational  commitments,  capability  requirements  and  other 
unexecutable  requirements  is  the  executable  requirement.  The  difference  between  the  executable 
requirement  and  the  funded  requirement  is  the  UDR  (Bachmann,  1994: 10).  This  study  will 
attempt  to  provide  the  framework  to  assess  weapon  system  availability  that  can  be  used  to  justify 
PDM  funding  for  the  UDR. 

Pro2rammed  Depot  Maintenance 

PDM  is  one  of  the  maintenance  activities  under  the  concept  of  Reliability  Centered 
Maintenance  (RCM)  which  all  DoD  agencies  are  required  to  follow  in  accordance  with  Defense 
Directive  4151.16  (Love,  1971:3).  The  goal  of  the  RCM  program  is  to  identify  potential  failures 
and  to  perform  maintenance  activities  to  prevent  the  failure.  The  maintenance  activities  include: 

1 .  Scheduled  inspection  of  an  item  at  regular  intervals  to  find  any  potential 
failures. 

2.  Scheduled  rework  of  an  item  at  or  before  some  specified  age  limit. 

3.  Scheduled  discard  of  an  item  at  or  before  some  specified  life  lime. 


4 


4.  Scheduled  inspection  of  a  hidden  function  item  to  find  functional  failures. 

(Love,  1971:8) 

These  actions  will  reduce  the  risk  of  catastrophic  failures  of  a  weapon  system  while  decreasing  the 
downtime  due  to  corrective  maintenance  in  the  field. 

The  program  begins  with  an  in-depth  failure  analysis  of  each  the  weapon  system’s  major 
components.  The  analysis  utilizes  both  engineering  data  as  well  as  operational  history.  A  failure 
in  these  components  is  then  categorized  as  either  an  “operational,  safety,  or  economic 
consequence”  (Love,  1971:6).  The  failure  is  further  categorized  as: 

1 .  Catastrophic:  A  failure  which  may  cause  death  or  weapon  system  loss. 

2.  Critical:  A  failure  which  may  cause  severe  injury,  major  property  loss,  or  major 
system  damage  which  will  result  in  mission  loss. 

3.  Marginal:  A  failure  which  may  cause  minor  injury,  property  damage  or  minor 
system  damage  which  will  result  in  delay,  loss  of  availability  or  mission 
degradation. 

4.  Minor:  A  failure  not  serious  enough  to  cause  injury,  property  damage  or 
system  damage,  but  which  will  result  in  unscheduled  maintenance  repair 
(corrective  maintenance). 

(Love,  1971:10) 

Maintenance  actions  are  then  scheduled  to  assess  the  failure  data  to  minimize  the  consequences  of 
failure.  PDM  deals  primarily  with  catastrophic  and  critical  failures,  but  is  also  scheduled  at 
specific  points  in  the  life  of  a  weapon  system  (mid-life  rebuild). 


Problem 

This  study  will  attempt  to  quantify  the  decrease  in  weapon  system  availability  in  the  field 
resulting  from  shortfalls  in  PDM  funding.  The  downtime,  or  unavailability,  of  a  weapon  system 
can  be  broken  down  into  two  distinct  components.  First,  the  weapon  system  experiences  a 
failure.  Then,  the  weapon  system  is  out  of  service  due  to  repair.  There  will  be  two  different 
approaches  used  to  model  these  components. 
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First,  a  Poisson  regression  model,  using  historical  failure  data  of  the  Marine  Corps’ 
Logistic  Vehicle  System  (LVS)  will  be  used  to  attempt  to  predict  the  expected  number  of  failures 
per  year.  Failures  will  be  broken  out  by  the  manufacturing  year  of  the  vehicle  for  the  LVS  and 
PDM  or  no-PDM  status. 

Second,  a  non-parametric  approach,  called  the  bootstrap  technique,  will  be  used  to 
estimate  the  expected  downtime  per  year  for  each  category  of  the  LVS.  The  bootstrap  technique 
samples  the  empirical  time-to-failure  distribution  to  obtain  a  failure  time.  It  then  samples  the 
empirical  service  time  distribution  to  obtain  a  service  time.  This  is  repeated  until  the  simulation 
clock  reaches  365  days.  The  empirical  distributions  are  estimated  using  the  Kaplan-Meier 
technique  (Leemis,  1995:257).  The  methodologies  for  both  approaches  are  further  developed  in 
Chapters. 

Scope 

This  study  will  focus  only  on  the  PDM  program  of  depot  maintenance.  The  analysis  will 
be  conducted  on  the  failure  data  of  the  Marine  Corps’  LVS.  The  LVS  truck  data  consists  of  5 
years  of  failure  data  (field  repair  only)  and  does  not  include  depot  repair.  It  was  extracted  from 
the  Marine  Corps’  Integrated  Maintenance  Management  System’s  (MIMMS)  data  base.  The 
other  data  sources  from  the  LVS  truck  include  data  obtained  from  the  depot  weapon  system 
equipment  manager  and  Oshcosh,  the  manufacturer  of  the  LVS. 
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II.  Literature  Review 


Recent  literature  indicates  that  there  are  several  studies  and  mathematical  models  which 
provide  insight  into  the  degradation  of  readiness  due  to  depot  maintenance  shortfalls.  The  Air 
Force  Funding  Availability  Multi-Method  Allocator  for  Spares  (FAMMAS)  modeled  the  decrease 
in  availability  due  to  PDM  shortfalls  on  a  component  level  with  aircraft.  Synergy  Incorporated 
proposed  a  methodology  to  model  aircraft  availability  proposing  a  Monte  Carlo  simulation.  A 
study,  “Depot  Maintenance  in  the  Air  Force:  How  Requirements  are  Determined  and  How  They 
Relate  to  Aircraft  Readiness  and  Sustainability,”  modeled  availability  of  aircraft,  while  deployed  in 
a  combat  scenario,  Avith  shortfalls  in  the  War  Reserve  Materiel  Kits  (WRSK)  due  to  lack  of  PDM 
funding.  Other  studies  calculated  the  optimal  PDM  cycle  which  minimizes  the  total  life-cycle 
cost  of  the  weapon  system.  They  include:  The  Army’s  Bradley  Fighting  Vehicle  Study  and  the 
Army’s  Ml -IP  Tank  Study.  A  thesis  by  Kephart  and  Roberts,  “Aligning  Demand  for  Spare  Parts 
with  their  Underlying  Failures  Modes,”  modeled  the  number  of  demands  using  multiple  linear 
regression  and  a  Poisson  process,  for  aircraft  reparables  at  the  work  unit  code  level.  Finally,  the 
bootstrap  method  offered  a  non-parametric  approach  to  estimate  the  number  of  failures  per  year 
and  the  downtime  per  year  using  empirical  failure  distributions. 


FAMMAS 

The  FAMMAS  model  assesses  readiness  impacts  of  unfunded  depot  maintenance.  It  is 
based  on  major  component  availability,  including  parts  procurement  and  initial  spares  funding. 
FAMMAS  focuses  more  on  the  exchangeable  component  repair  program  and  not  the  PDM 
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program.  It  illustrates  the  past  four  years,  the  current  year,  and  projects  three  fixture  fixnding 
years  to  accurately  model  actual  dollars  with  procurement  lags.  There  can  be  several  years 
between  a  component’s  order  time  and  receipt  time.  This  is  used  to  develop  the  funded 
requirements  /  total  requirements  ratio  for  the  projected  years.  FAMMAS  utilizes  the  following 
availability  marginal  return  model: 

3/=  1-ae'^’' 

where 

y  =  weapon  system  availability 
X  =  funding  ratio 
a,h  are  model  constants 

(FAMMAS;4-23) 

When  using  this  model,  the  “marginal  return  curves  for  funding  vs.  availability  should  incorporate 
simultaneous  treatment  of  buy,  repair,  and  initial  spares  funding  and  requirements”  (FAMMAS; 
4-22). 

FAMMAS  requires  both  the  historical  Total  Non  Mission  Capable  rate  due  to  Supply 
(TNMCS)  and  the  historical  Non  Mission  Capable  rate  due  Maintenance  (NMCM)  for  the 
weapon  system.  NMCM  rates  are  used  in  a  time  series  model  to  predict  a  mission  capable  rate. 
The  TNMCS  rate  calibrates  the  marginal  return  curve  for  the  weapon  system.  FAMMAS  fits  two 
points  to  obtain  the  constants  for  this  model. 

FAMMAS  utilizes  the  linear  relationship  of  1  -  TNMCS  and  aircraft  availability,  which  is 
output  from  the  Air  Force’s  Aircraft  Availability  Model  (AAM).  The  AAM  determines  the 
availability  of  an  aircraft  type  by  considering  the  availability  of  its  specific  components. 
Availability  ranges  from  zero  (not  available)  to  one  (always  available).  All  the  components  are 
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considered  to  work  independently,  therefore  aircraft  availability  is  the  product  of  its  component’s 
availability. 

The  AAM  prioritizes  the  buy  for  the  given  funding  level  and  “buys”  the  components  which 
maximizes  aircraft  availability.  Components  are  repaired  or  procured  only  when  they  increase 
aircraft  availability.  Therefore,  when  ftinding  decreases,  component  availability  decreases,  which 
in  return,  decreases  weapon  system  availability.  When  this  relationship  is  represented  as  a  general 
linear  model,  the  regression  constant  will  be  the  availability  with  no  funding,  which  is  the  first 
point.  The  second  point  is  the  base  year  availability  value  and  funding  ratio.  This  exponential 
relationship  reflects  a  marginal  decrease  in  availability  as  the  funding  ratio  decreases  and  exhibits 
diminishing  returns  as  the  ratio  increases.  This  model  deals  with  a  weapon  system  at  the 
component  level  and  can  not  be  applied  to  all  weapon  systems  in  the  DoD  inventory. 

Synergy  Incorporated 

Synergy  Incorporated,  an  analysis  company  located  in  Washington  D.C.,  did  a  preliminary 
study  on  the  UDR  problem  and  developed  a  proposed  methodology  to  solve  this  problem.  This 
study  was  never  funded.  The  plan,  “A  Discussion  of  Modeling  Support  for  PDM  Analyses  and 
Assessments,”  proposed  a  Monte  Carlo  simulation  to  determine  if  PDM  did  impact  readiness  and 
weapon  system  sustainability.  If  PDM  did  impact  readiness  and  weapon  system  sustainability. 
Synergy  wanted  to  generalize,  quantify,  and  credibly  project  PDM’s  effect  (Synergy,  1994: 1). 

“If  usefiil  and  verifiable  quantification  is  achieved,  can  it  be  codified  and  incorporated  into  a 
decision  support  architecture  and  used  to  defend  PDM”  (Synergy,  1994:1)? 

Synergy’s  approach  to  define  the  relationship  of  PDM  shortfalls  and  MC  rates  involves 
several  steps.  First,  Synergy  creates  a  critical  item  list  (depot  critical  items)  and  models  each 
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item’s  failure  rate,  estimating  the  Weibull  failure  distribution  parameters  for  each  critical 
component.  Next,  Synergy  will  perform  a  Monte  Carlo  study  to  estimate  aircraft  failure  rate  due 
to  each  PDM  critical  components  without  PDM.  This  “increase”  in  the  failure  rate  will  then  be 
translated  into  TNMCS  and  NMCM  rates.  Synergy  “anticipates  using  regression  analysis  on  time 
series  data  to  quantify  relationships  between  PDM  work  performed  and  weapon  system 
performance  ”  (Synergy,  1994:9).  Finally,  they  will  combine  these  rates  with  overall  rates  from  a 
model  like  the  FAMMAS  model  (Synergy,  1994:  18).  Like  the  FAMMAS  model,  this  approach 
deals  with  a  weapon  system  at  the  component  level  and  can  not  be  applied  to  all  the  weapon 
system’s  the  DoD’s  inventory. 

Depot  Maintenance  in  the  Air  Force;  How  Requirements  are  Determined  and  How  They 
Relate  to  Aircraft  Readiness  and  Sustainability 

O’Malley  and  Bachman  of  the  Logistics  Management  Institute  performed  a  study  on 
aircraft  availability  with  respect  to  a  shortage  of  serviceable  spare  parts.  In  1987,  large  cuts  in 
Depot  Purchased  Equipment  Maintenance  prompted  this  study,  but  it  focused  on  the 
exchangeable  component  repair  program  and  not  the  PDM  program  (DPEM)  (O’Malley  and 
Bachman,  1990:A-3).  They  studied  an  F-16A  24-aircraft  squadron  and  an  F-1 1  ID  18-aircraft 
squadron.  Each  squadron  was  analyzed  independently  and  the  study  was  limited  to  stock 
numbers  in  the  WRSK  kits.  In  order  to  get  two  years  of  projected  parts  for  depot  return,  they 
analyzed  the  D041  (component  peacetime  factors)  and  each  squadron’s  flying  hour  program 
(O’Malley  and  Bachman,  1990:3-2).  As  with  the  FAMMAS  model,  they  then  used  the  AAM  to 
prioritize  the  parts  repair,  but  only  used  a  fraction  of  the  total  funding  level  to  repair  all  the  parts, 
to  maximize  aircraft  availability.  They  used  the  Dyna-METRIC  (Dynamic  Multi-Echelon 


10 


Technique  for  Recoverable  Item  Control)  to  evaluate  the  resulting  degradation  of  availability  in 
the  scenario  that  the  squadron  deployed  with  a  depleted  WRSK  kit  (O’Malley  and  Bachman, 
1990:3-3).  They  found  that  each  WRSK  kit  could  support  the  planned  sortie  generation  for  both 
nominal  scenarios  (O’Malley  and  Bachman,  1990:A-6).  This  model  dealt  with  weapon  system 
component  spares  and  estimated  the  availability  of  aircraft  in  the  field  with  a  depleted  WRSK  kit. 
PDM  funding  for  spares  was  not  addressed  in  this  study. 

The  Bradley  Vehicle  Study 

The  Bradley  vehicle  study  was  established  to  determine  the  optimal  Inspect  and  Replace 

Only  when  Necessary  (IRON)  schedule.  IRON  is  part  of  the  Army’s  PDM  program.  The  study 

established  an  optimal  IRON  schedule  that  minimized  the  life  cycle  cost  of  the  weapon  system. 

The  Bradley  vehicle  study  extracted  data  from  316  Bradley  fighting  vehicles  during  16  Field 

Training  Exercises  (FTX)  conducted  at  the  National  Training  Center  (NTC),  Ft  Erwin,  CA.  The 

vehicles  were  divided  into  classes  by  mileage.  Each  interval  represented  a  multiple  of  750  miles. 

The  study  focused  on  class  averages: 

average  man  hours  =  sum  man  hours  per  class  /  sum  miles  per  class 
average  miles  =  sum  miles  per  class  /  sum  class  observations 

(Bradley:  1) 

The  Army  constructed  plots  for  maintenance  cost  vs.  mileage  and  man-hours  vs.  mileage  (where 
mileage  is  the  average  mileage  per  class).  The  maintenance  cost  and  mileage  relationship  can  be 
represented  as: 

y  —  ox* 

where 

y  =  maintenance  cost  (average  maintenance  cost  per  class 
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X  =  mileage  (average  miles  per  class) 
a,b  are  constants 

(Bradley:  14) 

They  further  established  the  relationship  between  Man-hours  and  mileage  as  the  linear  model: 

j  =  po  +  Piv 

where 

y  -  man-hours  (average  man-hours  per  class) 

X  =  mileage  (average  miles  per  class) 

(Bradley:  16) 

Both  models  were  statistically  significant.  Utilizing  the  established  relationship  between 
maintenance  cost  and  mileage,  the  optimal  IRON  cycle  was  2  inspections  for  the  lifetime  of  this 
weapon  system.  This  is  found  by  the  difference  in  the  cost  curves  over  time  (the  difference  is  the 
savings)  of  no  IRON’S  with  one  IRON,  two  IRON,  and  so  forth.  This  assumes  that  an  IRON 
inspection  returns  the  vehicle  to  a  “zero”  mileage  vehicle.  The  savings  is  optimized  with  two 
IRON  inspections.  This  study,  however,  does  not  address  the  impact  of  the  PDM  inspection  on 
weapon  system  availability. 

Army  Materiel  Systems  Analysis  Activity  Study 

The  Army  Materiel  Systems  Analysis  Activity  (AMS AA)  conducted  a  similar  study,  but 
AMSAA  first  wanted  to  validate  the  IRON  program.  If  the  IRON  program  significantly  improved 
availability,  then  they  would  develop  an  IRON  schedule  that  minimized  life  cycle  cost.  The  study 
used  M-lIP  tanks  during  19  FTX  exercises  at  the  NTC.  Fourteen  tanks  entered  Reliability 
Centered  Inspect  and  Replace  Only  when  Necessary  (RC-IRON)  maintenance  and  46  non-RC- 
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IRON  tanks  participated  in  the  exercise.  The  study  found  that  the  IRON  program  sigmficantly 
reduced  part  replacements  per  mile,  essential  maintenance  actions  per  mile,  total  maintenance 
actions  per  mile,  man-hours,  and  support  cost  indicators.  They  did  not,  however,  test  for 
significance  between  the  availability  of  the  IRON  and  non-IRON  tanks.  The  same  methodology  as 
the  Bradley  study  was  used  to  determine  the  optimal  IRON  schedule  (Fox  and  Chung,  1991). 


Aligning  Demand  for  Snare  Parts  with  their  Underlying  Failure  Modes 

A  thesis  by  Kephart  and  Roberts  attempted  to  predict  the  number  of  demands  for  aircraft 
reparables  at  the  work  unit  code  level.  They  used  a  three  phase  methodology  which  included  a 
multiple  linear  regression,  a  Poisson  regression,  and  lastly,  fitting  a  Poisson  process.  The 
multiple  regression  model  in  their  study  used  cumulative  flying  hours  and  number  of  sorties  to 
predict  the  number  of  demands  or  maintenance  actions  of  a  work  unit  code. 

j  =  Po  +P,x,  +P2X,  +p3xf  +p,x,"  -i-P5X,x, 


where 


Xi  =  commutative  flying  hours 
X2  =  number  of  sorites 

(Kephart  and  Roberts,  1995:3-9) 

For  the  Poisson  regression,  Kephart  and  Roberts  used  three  different  link  functions  to  predict  the 
number  of  demands: 


lt(-*^,P)  =  Po  +Pi^/ 


lt(^,P)  =  3o  +Pl^2 


ft  (•^;  P)~Po  "*"32^2 

(Kephart  and  Roberts,  1995:3-11) 
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The  Poisson  regression  and  the  Poisson  process  analysis  is  detailed  in  Chapter  3.  The  results  of 
the  Kephart  and  Roberts  study  showed  a  limited  relationship  with  demands  and  the  independent 
variables.  They  attributed  the  inability  to  accurately  model  demands  to  the  erratic  nature  of  the 
demands  (Kephart  and  Roberts,  1995:5-4).  In  other  words,  the  failure  rate  was  not  stationary. 
They  concluded  that  the  “work  unit  code  level  demand/maintenance  action  estimates,  which  were 
based  on  flying  hours  and  sorties  from  past  demands,  were  poor  estimators  of  future 
demands/maintenance  actions”  (Kephart  and  Roberts,  1995:5-4).  The  Poisson  regression  model 
will  also  be  used  in  this  study  to  model  the  failures  per  year  of  the  LVS. 

Bootstrap  Technique 

Similar  to  the  Monte  Carlo  model  suggested  by  Synergy,  the  bootstrap  technique 
(described  in  The  Introduction  to  the  Bootstrap  by  Efron  and  Tibshirani)  is  a  non-parametric 
technique  that  can  model  number  of  failures  per  year  and  downtime  per  year  for  a  weapon  system. 
This  technique  is  describe  in  detail  in  Chapter  3.  The  bootstrap  technique  simulates  a  process 
which,  in  this  case,  is  a  failure  process.  The  technique  entails  a  random  sampling  with 
replacement  from  a  process’s  empirical  distribution  to  obtain  the  statistic  of  interest.  The  Kaplin- 
Meier  technique  described  by  Leemis  in  Reliability  Probabilistic  Models  and  Statistical  Methods 
can  be  used  to  derive  the  empirical  distributions  for  time  to  failure  and  service  time.  This 
technique  is  also  described  in  more  detail  in  Chapter  3. 

Summary 

Little  research  has  been  done  to  show  the  effect  of  PDM  on  overall  weapon  system 
availability,  and  the  research  that  has  been  conducted  has  been  limited  to  large,  complex  weapon 
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systems  at  the  component  level.  Many  weapon  systems  in  the  DoD  inventory  are  not  tracked  at 
the  component  level,  therefore,  component  level  research  can  not  be  applied  to  all  weapon 
systems.  This  research  effort  will  be  conducted  in  two  phases.  First,  the  failures  per  year  will  be 
modeled  as  a  Poisson  process  using  the  Poisson  regression,  and  secondly,  the  number  of  failures 
per  year  and  downtime  per  year  will  be  estimated  using  the  bootstrap. 
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in.  Methodology 


The  purpose  of  this  study  is  to  determine  the  loss  of  availability  of  a  weapon  system  when, 
due  to  lack  of  funding,  PDM  could  not  be  performed.  If  the  PDM  program  reduces  the  number 
of  failures  per  year  per  vehicle  and  reduces  the  downtime  per  failure,  the  increase  in  availability 
due  to  the  PDM  program  can  possibly  be  estimated.  This  chapter  will  outline  the  research 
questions,  data  collection,  and  proposed  methodologies  used  to  determine  weapon  system 
availability. 

Research  Questions 

In  order  to  determine  the  increase  in  availability  of  a  weapon  system  due  to  PDM,  the 
following  questions  must  be  answered: 

1 .  Does  PDM  reduce  the  number  of  failures  per  year? 

2.  Does  PDM  reduce  the  amount  of  downtime  once  the  weapon  system  has  failed? 

If  PDM  does  reduce  the  number  of  failures  per  year  and  downtime  per  year,  the  increase  in 
availability  due  to  the  PDM  program  can  then  be  modeled  and  quantified. 

Data  Collection 

Weapon  system  availability  requires  collection  of  failure  data  on  a  weapon  system  to  be 
studied.  More  specifically,  the  weapon  system’s  serial  numbers,  manufacturing  dates,  and  failure 
data  must  be  known  for  the  entire  inventory.  This  study  will  focus  on  the  Marine  Corps  MK-48  or 
LVS.  Unfortunately,  the  Marine  Corps  did  not  have  a  comprehensive  list  of  their  LVS  inventory. 
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A  partial  list  containing  1077  of  the  1680  of  the  LVS’s  and  their  ship  dates  was  obtained  from 
Oshcosh,  the  manufacturer  of  the  LVS.  The  serial  numbers  were  assigned  by  Oshcosh  in 
ascending  order.  The  number  of  LVS’s  issued  per  year  was  obtained  from  the  LVS  item 
manager.  The  steps  used  to  determine  the  valid  serial  numbers  and  there  manufacturing  dates 
were: 

1.  Generate  a  list  of  serial  numbers  from  the  Marine  Corps’  data  base. 

2.  Remove  serial  numbers  that  were  not  the  standard  six  digit  numeric  codes. 

3.  Compare  the  updated  list  Avith  the  list  obtained  from  Oshcosh. 

4.  Remove  invalid  serial  numbers  from  the  first  1 077  LVS  serial  numbers  from 
updated  list. 

5.  From  the  remaining  serial  numbers,  count  down  and  assign  a  manufacturing 
date  to  each  vehicle. 

6.  Because  the  serial  numbers  are  ascending,  but  are  not  sequential,  try  and 
identify  correct  serial  numbers  from  those  serial  numbers  that  were  removed. 

7.  Correct  invalid  serial  numbers  from  data  base  (100  serial  numbers  were 
corrected). 

The  failure  data  were  collected  by  the  Marine  Corps  base  s  maintenance  personnel. 

One  data  record  was  collected  for  each  failure.  The  data  collected  for  this  research  study 
included  date  the  LVS  was  received,  the  date  the  LVS  was  released  (date  closed),  echelon 
of  maintenance,  and  mileage  at  the  time  of  failure. 

Unfortunately,  this  research  study  could  not  use  mileage  at  the  time  of  failure.  More  than 
one-third  of  the  mileage  data  in  the  database  were  incorrect.  The  LVS  has  two  usage  meters,  one 
to  measure  mileage  and  the  other  to  measure  hours  of  use.  The  Maintenance  Directorate 
responsible  for  the  LVS  stated  that  either  number  could  have  been  entered  into  the  database  and 
was  unreliable.  Many  other  mileage  readings  were  either  0  or  999,999.  Therefore,  this  part  of 
the  database  was  too  corrupt  to  use  in  this  research  study. 

The  echelon  of  maintenance  indicated  the  severity  of  the  failure,  as  well  as  the  type  of 
maintenance  performed  on  the  vehicle.  The  manner  in  which  it  was  recorded  introduced  another 
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problem  in  the  data.  Once  an  LVS  failed,  an  entry  was  made  in  the  database.  If  the  shop  could 
not  fix  the  problem  due  to  the  severity  of  the  failure,  the  LVS  was  transferred  to  another 
maintenance  activity  and  another  entry  for  the  same  failure  was  made  into  the  database.  Usually 
the  previous  entry  was  not  closed  out.  Once  the  second  maintenance  activity  fixed  the  failure,  the 
second  entry  was  closed,  and  the  LVS  was  either  sent  back  to  the  previous  shop  or  directly 
released  back  into  service.  If  the  LVS  was  released  directly  into  service,  the  second  maintenance 
shop  notified  the  first  shop  before  it  was  released  and  the  first  entry  was  closed.  If  the  LVS  was 
sent  back  to  the  previous  shop,  the  first  entry  was  closed  upon  release.  At  first  glance,  these 
entries  appear  to  be  multiple  failures,  but  in  actuality,  these  entries  only  represent  one  failure  with 
a  downtime  of  latest  date  closed  minus  the  earliest  date  received. 

In  order  to  analyze  availability  of  the  LVS  that  went  to  PDM,  the  serial  number  and  dates 
received  into  PDM  of  the  LVS’s  that  went  through  programmed  maintenance  must  be  collected. 
The  Marine  Corps  did  not  collect  this  information  on  the  LVS.  However,  the  item  manager  did 
keep  a  log  of  all  vehicle  serial  numbers  and  the  year  and  month  of  a  PDM  request.  The  time 
between  the  request  and  actual  program  maintenance  was  not  known,  but  it  is  assumed  to  have  a 
zero  lag  time.  Mileage  data  and  the  amount  of  time  a  weapon  system  stayed  in  PDM  were  also 
not  collected. 

Maintenance  data  were  collected  from  the  first  quarter  of  1990  to  the  third  quarter  of 
1995.  This  included  22,1 15  data  entries.  After  the  invalid  serial  numbers  were  removed,  the  file 
consisted  of  20,830  data  entries  with  1567  valid  serial  numbers.  Once  the  data  entries  were 
corrected  to  account  for  the  LVS’s  being  transferred  from  one  maintenance  activity  to  another, 
the  data  file  consisted  of  1 0, 1 54  entries.  A  FORTRAN  program  was  written  to  obtain  the  number 
of  failures  per  year  for  each  vehicle  for  the  Poisson  regression.  A  failure  was  counted  as  a  no- 
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PDM  failure  if  the  vehicle  did  not  go  to  PDM  before  the  failure.  A  failure  was  counted  as  a  PDM 
failure  if  the  failure  occurred  after  a  vehicle  went  to  PDM.  When  a  failure  occurred  in  a  PDM 
year,  this  became  a  partial  no-PDM  year  and  a  partial  PDM  year.  An  expected  number  of  no- 
PDM  failures  and  an  expected  number  of  PDM  failures  had  to  be  calculated  as  if  it  was  a  full  year. 
The  expected  number  of  no-PDM  failures  was  equal  to  the  number  of  failures  before  PDM 
divided  by  the  fraction  of  the  year  the  vehicle  was  in  a  no-PDM  status.  The  expected  number  of 
PDM  failures  was  equal  to  the  number  of  failures  after  PDM  divided  by  the  fi-action  of  the  year 
the  vehicle  was  in  a  PDM  status. 

Proposed  Models 

Two  proposed  methodologies  will  attempt  to  define  the  relationship  between  availability 
and  PDM.  First,  a  Poisson  regression  will  attempt  to  quantify  the  relationship  between  PDM  and 
the  number  of  failures  per  year  and  predict  the  number  of  failures  per  year. 

The  second  phase  of  this  research  study  is  non-parametric  in  nature.  It  involves  the 
bootstrap  technique  described  in  The  Introduction  to  the  Bootstrap  by  Efron  and  Tibshirani.  The 
bootstrap  technique  is  a  database  simulation  developed  to  make  certain  statistical  inferences” 
(Efron  and  Tibshiranti,  1993  :1).  This  technique  can  be  used  to  estimate  sample  means,  standard 
deviations,  confidence  intervals,  and  so  forth”  (Efron  and  Tibshirani,  1993:vii).  In  general,  the 
bootstrap  technique  will  randomly  draw  samples  with  replacement  from  a  population  represented 
by  empirical  distributions.  From  this  sample,  a  statistic  of  interest  is  then  calculated.  The 
experiment  is  replicated  to  define  confidence  intervals  around  the  statistic  of  interest.  This  study 
will  randomly  draw  a  time  to  failure  from  its  empirical  distribution  and  a  downtime  from  its 
empirical  distribution  repeating  the  process  until  the  total  number  of  days  exceeds  365.  This 
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procedure  will  be  done  for  eEch  vehicle  in  the  fleet  to  obtain  one  bootstrap  replication.  The 
statistics  of  interest  are  the  downtime  per  year  per  vehicle,  number  of  failures  per  year  per  vehicle, 
and  downtime  the  extending  into  the  following  year  per  vehicle.  Confidence  intervals  will  be 
calculated  using  2000  bootstrap  replications.  The  Kaplin-Meier  technique  will  generate  an 
empirical  survival  curve,  S(t).  The  empirical  distribution  of  the  time  to  failure  is  then  1  -  S(t). 

Poisson  Regression.  Since  the  number  of  failures  per  year  is  a  discrete  response,  this 
violates  the  normality  assumption  of  the  error  terms  for  multiple  linear  regression.  The  number  of 
failures,  N(t),  as  a  function  of  regressor  variables,  however,  can  be  modeled  assuming  a  Poisson 
process  (Myers,  1986:333).  The  assumptions  for  a  Poisson  process  are  introduced  in  Stochastic 
Processes  by  Ross. 

A  stochastic  process  {N(0,  ^^0}  is  said  to  be  a  counting  process,  if  N(0 
represents  the  total  number  of  “events”  that  have  occurred  up  to  time  t. 

Hence,  a  counting  process  N(0  must  satisfy: 

(i)  N(0>0 

(ii)  N(r)  is  integer  value 

(iii)  s<t,  then  N(5)  <  N(0 

(iv)  For  5  <  r,  N(0  -  N(5)  equals  the  number  of  events  that 
have  occurred  in  interval  (s,t] 

(Ross,  1983:31) 

The  counting  process  {N(0,  ^^0}  is  said  to  be  a  Poisson  process  having 
rate  X,  A,  >  0,  if: 

(i)  N(0)  =  0 

(ii)  The  process  has  independent  increments 

(iii)  The  number  of  events  in  any  intervals  length  t  is  Poisson 
distributed  with  mean  Xt.  That  is,  for  all  5,  t  >  0, 

^(^  0" 

P{Nit  +  s)-  N(s)  =  ri}  =  e  —— 
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where 


P{  N{t  +  s)-  N{s)  =  n)  represents  the  probability  on  n  failures  in  time 
interval  t 

(Ross,  1983:31) 

Poisson  regression  assumes  the  number  of  failures  per  year  does  not  change  independently  from 
one  data  point  to  the  next  but  depends  on  a  function  of  regressor  variables,  iJ.(x,,P)  (/  -1,2, . .  ,n), 
where  n  is  the  number  of  regressor  variables.  This  function  is  often  referred  to  as  the  link  function 
and  must  always  be  non-negative  (Myer,  1990:334).  The  proposed  link  function  in  this  study  is. 
H(Ar,|5)  =  p„  +  +  PjJT,  +  +  Ps^i  +  P.^;. 


where 


p  (x,  P )  is  the  expected  number  of  failures  per  year 
f  1,  if  the  LVS  went  to  PDM 

X  ^  i 

‘  [0,  otherwise 

=  years  since  last  PDM 

[  1,  if  the  LVS  was  manufactured  in  85 
1 0,  otherwise 


( I,  if  the  LVS  was  manufactured  in  86 

X 

0,  otherwise 


f  1, ;/  the  LVS  was  manufactured  in  87 
1  0,  otherwise 


^6 


\,  if  the  LVS  was  manufactured  in  88 
0,  otherwise 


If  the  LVS  was  manufactured  in  89,  x,  -  0,  /  -  3  . .  6. 
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The  Poisson  regression  model  assumes  a  stationary  failure  rate.  In  order  to  test  this 
assumption,  a  separate  Poisson  regression  will  attempt  to  show  the  year  of  the  failure  is  not 
significant.  The  proposed  link  function  for  this  Poisson  regression  is: 

P(a:,P)  =  Po  +Pi^1  +P2^2  +p3^3  +p4^4  +p5^5  +p6^6  +P7^7  +P8^8  +P9^9 


where 


p  (jc,  P  )  is  the  expected  number  of  failures  per  year 

f  1,  if  the  LVS  went  to  PDM 
1 0,  otherwise 


{ 1,  if  the  LVS  was  manufactured  in  85 
[O,  otherwise 

if  the  LVS  was  manufactured  in  86 
[0,  otherwise 


if  the  LVS  was  manufactured  in  87 
[O,  otherwise 


X, 


\,  if  the  LVS  was  manufactured  in  88 
0,  otherwise 


^6 


1,  if  the  failure  ocatred  in  1991 
0,  otherwise 


Xj 


\,  if  the  failure  occured  in  1992 
0,  otherwise 


I,  if  the  failure  occured  in  1993 
0,  otherwise 


Xg 


\,  if  the  failure  occured  in  1994 
0,  otherwise 
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If  the  LVS  was  manufactured  in  89,  x,  =  0,  i -2  . .  5. 

If  the  failure  occurred  in  1995,  x,.  =  0,  /  =  6  .  .  9. 

To  ensure  that  the  number  of  failures  per  year  per  vehicle  follow  a  Poisson 
distribution,  the  number  of  failures  per  year  per  group  will  be  subjected  to  a  Chi-Squared 
goodness  of  fit  test. 

The  test  statistic  for  the  Chi-Squared  test  is: 


where 

k  is  the  number  of  intervals 
Nj  is  the  number  of  x,  ’5  in  the  jth  interval  [aj-i,aj) 

pj  is  2;p(x,.) 

aj.iSXjSaj 

p(Xi )  is  the  probability  density  function  of  the  Poisson  distribution 

(Law  and  Kelton,  1991  ;382) 

The  hypothesis  test  for  goodness  of  fit  is  for  the  Poisson  distribution  is: 

Ho:  number  of  failures  per  year  is  Poisson 
Ha:  not  Poisson 

We  reject  Ho  if  the  test  statistic  is  greater  than  xLm-i,i-a 
where 

m  is  the  number  of  parameters  to  be  estimated 


a=.05 


(Law  and  Kelton,  1991 :383) 
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If  Ho  is  not  rejected,  the  failure  data  for  the  LVS  can  be  assumed  to  follow  a  Poisson 

distribution.  This  supports  the  use  of  a  Poisson  regression  model  on  the  LVS  failure  data. 

YEARS  SINCE  PDM  indicates  the  years  since  an  LVS’s  manufacturing  date  or 

since  PDM.  Since  there  can  be  multiple  failures  in  a  given  year,  the  years  since  PDM 

variable  had  to  be  aggregated  into  one  value.  The  failures  are  assumed  to  occur  uniformly 

throughout  the  year  and  the  mid-point  of  that  year  (182.5  days)  was  used  to  calculate  the 

years  since  PDM.  If  a  failure  occurred  in  a  PDM  year,  the  years  since  PDM  was 

calculated  as  the  proportion  of  the  year  in  PDM  status.  To  ensure  that  the  failures  in  a 

given  year  occur  uniformly  throughout  the  year,  the  failures  will  be  subjected  to  a  Chi- 

Squared  goodness  of  fit  test.  The  hypothesis  test  for  goodness  of  fit  for  the  uniform  is; 

Ho;  failures  are  uniformly  distributed  through  out  the  year 
Ha;  not  uniform 

We  reject  Ho  if  the  test  statistic  is  greater  than  xLm-i,i-a 
where 

m  is  the  number  of  parameters  to  be  estimated 


a=.05 


If  Ho  is  not  rejected,  failures 


(Law  and  Kelton,  1991  ;383) 

for  the  LVS  can  be  assumed  to  occur  uniformly  throughout  the  year. 


This  also  suggests  that  the  expected  day  in  which  a  failure  occurs  will  be  on  the  182.5  day  of  the 


year.  For  the  variable  YEARS  SINCE  PDM,  the  days  in  which  failures  occur  can  be  aggregated 
into  one  time  of  failure  occurring  on  the  182.5  day  of  the  year. 

Since  this  model  uses  a  link  function  to  predict  the  number  of  failures,  the  model  for  the  a 

Poisson  process  with  regressor  variables  is; 
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PCv,;P  )-e-'' 


where 


P(y,. ;  P  )  represents  the  probability  of  failures  occurring 
at  by  time 

|a(A:,. ,  P )  represents  the  Poisson  mean  which  replaces  |i  in 
the  standard  Poisson  process 

P  represents  a  vector  of  parameters  to  be  estimated 

jc,  represents  a  regressor  variable 

t,  represents  time 

represents  the  number  of  failures 

(Myer,  1990:334) 


The  model  for  the  Poisson  mean  is: 


where 


Pi  =  t,p(x,  ,p) 


/=1,  2,  .  .  ,n 


(Myer,  1990:334) 

In  this  study,  t,  will  always  equal  one  because  the  proposed  Poisson  regression  is  modeling 
failures  per  year. 

The  Poisson  regression  estimates  P  by  maximizing  the  likelihood  function.  The  likelihood 
function  for  the  Poisson  regression  is: 


L(T>P  )  =  np(>'.’P) 

i=l 
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(Myer,  1990:334) 


)  =  fl 


i=l 


The  procedure  used  to  find  the  maximum  likelihood  estimators  for  Poisson  regression  is 
iteratively  reweighted  least  squares  (Myer,  1990:335).  The  likelihood  equations  to  be  solved  for 
the  Poisson  regression  are: 


glnL(j^,P) 

ap 


or, 


z 


p(^i  J) 


ap(A:.,P) 

ap 


=  0 


(Myer,  1990:334) 

Siegel  notes  that  this  model  relates  the  link  function  to  the  natural  logarithm  of  the  number  of 
failures  per  year  (Siegel,  1 992 : 1 80). 

The  measure  for  lack  of  fit  uses  a  statistic  called  the  deviance  (McCullagh  and 
Nelder,  1992: 197).  Deviance  is  defined  as: 


D(T,P)  =  2-2 


(McCullagh  and  Nelder,  1992: 197) 


The  deviance  is  approximately  distributed  as  a  ^  distribution  with  n-k-1  degrees  of 
freedom  where  n  is  the  sample  size  and  k  is  the  number  of  parameters  to  be  estimated  plus  one. 
The  hypothesis  test  to  measure  the  lack  of  fit  is: 


Ho:  model  is  Poisson 
Ha',  not  Poisson 

We  reject  Hoif  D(j?,//)  is  greater  than  xLk-i.i-a 
where 
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a=  05 


(Myer,  1990:324) 


If  Ho  is  not  rejected,  than  the  model  can  be  assumed  to  be  Poisson.  This  will  provide  statistical 
significance  to  the  results  of  the  tests  on  the  individual  coefficients. 

The  individual  coefficients  can  be  tested  similarly  to  a  t-test  in  multiple  linear  regression, 
but  they  also  follow  a  distribution  with  one  degree  of  freedom.  The  test  statistic  is  ; 


where 

7  =  0,  1, .  .  k-\ 

bi  =  the  parameter  estimate  for  P, 

Ci  =  the  standard  error 

The  hypothesis  test  for  the  individual  coefficients  is; 

Ho:  p,  =  0 

Ha;  P,  0  given  the  rest  of  the  coefficients  are  in  the  model 

.  .2 

We  reject  Ho  if  —  is  greater  than  Xa.\ 

where 


a=.05 


(Siegel,  1992:181) 


If  Ho  is  rejected,  the  coefficients  can  be  assumed  to  be  significantly  different  than  zero,  and  it  can 
be  assumed  that  the  variable  corresponding  to  the  coefficients  contributes  to  the  number  of 
failures  per  year. 

An  F-test  will  test  if  the  failure  rate  for  PDM  vehicles  is  significantly  different  than  the 
failure  rates  for  no-PDM  vehicles.  The  test  statistic  for  the  F-test  is: 
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where 


X  , 

r. 


X,,  >  X2 

The  hypothesis  test  the  difference  of  two  failure  rates  is: 

Ho:  X2 

Ha:  Xi 't-  Xi 

X  , 

We  reject  Ho  if  — is  greater  than  F  „ 

_  2r,,2r,,l-- 


where 


a=.05 


(Leemis,  1995:210) 

If  Ho  is  rejected,  the  failure  rates  for  PDM  and  no-PDM  can  be  assumed  to  be  statistically 
different.  This  also  suggests  that  the  failure  rates  for  PDM  are  significantly  lower  than  no- 
PDM  failure  rates. 


Bootstrap  Technique.  “The  bootstrap  method  is  a  computer-based  method  for  assigning 
measures  of  accuracy  to  statistical  estimates”  (Efron  and  Tibshirani,  1993:10).  In  this  study,  the 
statistical  estimate  will  be  downtime  per  vehicle  per  year.  Vehicles  will  be  differentiated  by  their 
PDM  status  and  manufacturing  year.  The  process  involves  independently  sampling  with 
replacement  from  two  empirical  probability  distributions:  F  (status,  year  group)  to  obtain 
bootstrap  samples  x*  for  time  to  failure  and  G  (status,  year  group)  to  obtain  bootstrap  samples  y 
for  downtime  after  failure.  The  Probability  Distributions,  F  (status,  year  group)  and  G  (status, 
year  group),  will  be  1-S(t),  where  S(t)  is  the  survivor  functions  estimated  using  the  Kaplin-Meier 
technique  which  will  be  addressed  later  in  this  section.  The  sampling  will  end  when  the  total  days 
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exceed  365  days.  This  will  be  repeated  k  times  where  k  is  the  number  of  trucks  in  the  simulation. 
In  this  study,  k  is  1000  trucks.  The  statistics  of  interest  are  the  average  downtime  per  vehicle, 
average  number  of  failures  per  vehicle,  and  average  downtime  in  the  following  year  per  vehicle  for 
a  single  year.  Each  statistic  of  interest  is  considered  one  bootstrap  sample  and  is  defined  as: 


^downtime 


^  number  of  failures 
: - 


downtime  in  the  next  year 


where 

b  is  the  index  for  the  b*  bootstrap  sample 

(Efron  and  Tibshirani,  1993:47) 

The  number  of  replications  of  the  bootstrap  sample  should  be  chosen  to  minimize  the 
standard  error.  Unfortunately,  for  most  statistics  there  is  no  limiting  value  of  the  standard  error 
(Efron  and  Tibshirani,  1 993 : 1 5).  Efi-on  and  Tibshirani  state  that  there  seldom  are  more  than  200 
replications  needed,  but  more  replications  are  required  for  bootstrap  confidence  intervals  (Efron 
and  Tibshirani,  1993:53).  More  specifically,  the  number  of  replications  increases  by  a  factor  of 
ten  for  a  confidence  interval  (Efron  and  Tibshirani,  1993:15).  Thus,  the  number  of  replications  in 
this  study,  B,  is  2000. 

The  bootstrap  statistic  is  defined  as: 
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'Ls(x-‘) 

S(.x-)  =  ^ - 

(Efron  and  Tibshirani,  1993  :47) 
This  bootstrap  statistic  represents  the  steady  state  values  obtained  from  bootstrap  sample. 
The  standard  error  is  defined  as; 


se 


boot 


1 


B 

V 

la 

6=1 

B-\ 

where 

IS(x-‘) 

sw  is  - 

(Efron  and  Tibshirani,  1993:47) 


The  90%  confidence  interval  is  defined  as: 

S(x*)  +  1.645(se,^.) 

(Efron  and  Tibshirani,  1993:153) 

Kanlan-Meier  Survival  Curve.  The  typical  non-parametric  estimation  of  a  survivor 


curve  is: 


*^(0  = 


Njt) 

n 


t  >0 


where 

N{t)  is  the  number  of  failures  at  time  t 
n  is  total  number  of  failure  times 

(Leemis,  1995:254) 
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This  estimation  will  be  used  to  estimate  the  empirical  downtime  distributions.  The  step  function 
has  a  downward  step  of  \/n,  and  all  n  values  have  equal  step  mass.  Ties  are  easily  corrected  by 
changing  the  step  size  to  d/n  where  d  is  the  number  of  ties.  This  method  does  not  take  into 
account  right-censored  data,  indicating  vehicles  which  did  not  fail  before  the  study  concluded. 
The  Kaplan-Meier  offers  a  non-parametric  technique  which  handles  right-censored  data.  The 


Kaplan-Meier  technique  is  defined  as: 


s(o=  n 

/eR(t)' 


where 

dj  denotes  the  number  of  observed  failures  at  time  y,; 

j=\,2,..k 

k  denotes  the  number  of  failure  times 
yj  denotes  the  7*  failure  time 

nj  denotes  the  number  of  items  on  test  just  before  time  yj  (This 
includes  any  values  that  are  cencored  at  time  yj) 

'  denotes  the  set  of  all  indexes  of  items  that  are  at  risk  just 
before  time_yy  =  ( 

(Leemis,  1995:256) 

The  Fleming  and  Harrington  log-rank  test  and  the  Peto-Peto  modification  of  the  Fleming 
and  Harrinton  log-rank  test  compare  the  survivor  functions  of  two  different  groups  (Venables  and 
Ripley,  1994:  273).  Suppose  i  groups,  with  survival  times,  have  d  observed  deaths  at  time  tj 
(where  j  is  the  number  of  unique  death  times  from  the  i  groups).  The  test  statistic  for  the  log-rank 
test  is: 

(O  -  E)^  wV'  (O  -  E) 

where 
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wV  is  the  weighted  covariance  matrix  of  (O  -  E) 
w  is  one  for  the  log-rank  test 

w  is  the  Kaplan-Meier  product  limit  estimator  for  the  Peto-Peto 
modification 

(Venables  and  Ripley,  1994;  273) 

The  test  statistic  is  distributed  as  a  xli-i  •  The  hypothesis  test  is: 

Ho:  the  two  survivor  curves  are  the  same 
Ha;  the  two  survivor  curves  are  not  the  same 

We  reject  Ho  if  log-rank  test  statistic  is  greater  than  x^i-i 
where 

a=.05 

If  Ho  is  rejected  for  all  the  survivor  curves,  the  PDM  and  no-PDM  survivor  curves  are  statistically 
different.  This  further  suggests  that  the  PDM  vehicles  remain  in  service  longer  between  failures 
and  no-PDM  vehicles  remain  in  the  maintenance  facility  longer  after  failure. 

The  techniques  described  in  this  chapter  will  be  implemented  on  the  LVS  data  to  attempt 
to  estimate  the  availability  of  the  PDM  and  non-PDM  vehicles. 


32 


IV.  Results  and  Analysis 


This  chapter  will  outline  the  results  from  the  methodologies  introduced  in  Chapter  3.  The 
first  section  will  detail  the  results  of  the  Poisson  regression,  beginning  with  a  discussion  of  the 
assumptions.  Next,  the  results  of  the  Poisson  distribution  goodness  of  fit  for  the  number  of 
failures  per  year  and  the  results  of  the  uniform  distribution  goodness  of  fit  for  the  failures 
occurring  within  each  year  will  be  presented.  The  results  of  the  Poisson  regression  will  then  be 
presented  with  a  detailed  analysis  of  the  residuals.  The  second  section  will  deteul  the  results  of  the 
bootstrap  technique.  First,  the  Kaplan-Meier  results  will  be  presented  and  finally  the  results  of  the 
bootstrap  will  be  discussed  with  its  implications. 

Poisson  Regression 

To  assume  the  number  of  failures  per  year  per  group  is  a  Poisson  process,  the  number  of 
failures  per  year  must  be  Poisson  distributed.  To  test  this  assumption,  a  Chi-Squared  Goodness 
of  Fit  test  was  conducted  using  Best  Fit.  The  results  of  the  test  are  given  in  Table  1 .  The  graphs 
depicting  the  number  of  failures  per  year  and  their  parametric  Poisson  distribution  are  included  in 
Appendix  A. 

The  number  of  failures  per  year  for  all  the  groups  represented  in  Table  1,  with  the 
exception  of  the  year  1989  no-PDM  group,  fail  to  reject  the  null  hypothesis  and  may  be  assumed 
to  follow  a  Poisson  distribution.  It  is  not  surprising  that  the  year  1989  no-PDM  group  rejected 
the  null  hypothesis.  The  1989  year  group  has  a  small  sample  size,  only  31  vehicles  for  1989,  and 
these  vehicles  are  relatively  new  compared  to  the  other  vehicles.  In  general,  the  1989  year  vehicles 
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Table  1 


Goodness  of  Fit  Results  for  the  Poisson  Distribution 


GROUP 

P-VALUE 

HYPOTHESIS  TEST 

RESULT 

a =.05 

85  no-PDM 

5.51 

0.788 

Ho  POISSON 

Ha:  NOT  POISSON 

DO  NOT  REJECT  Ho 

85  PDM 

7.19 

0.617 

Ho  POISSON 

Ha!  NOT  POISSON 

DO  NOT  REJECT  Ho 

86  no-PDM 

11.22 

0.261 

Ho  POISSON 

Ha:  NOT  POISSON 

DO  NOT  REJECT  Ho 

86  PDM 

4.44 

0.88 

Ho  POISSON 

Ha:  NOT  POISSON 

DO  NOT  REJECT  Ho 

87  no-PDM 

7.18 

0.618 

Ho  POISSON 

Ha:  NOT  POISSON 

DO  NOT  REJECT  Ho 

87  PDM 

m 

0.946 

Ho  POISSON 

Ha:  NOT  POISSON 

DO  NOT  REJECT  Ho 

88  no-PDM 

0.37 

0.999 

Ho  POISSON 

Ha:  NOT  POISSON 

DO  NOT  REJECT  Ho 

88  PDM 

15.64 

0.075 

Ho  POISSON 

Ha:  NOT  POISSON 

DO  NOT  REJECT  Ho 

89  no-PDM 

67.93 

0 

Ho  POISSON 

Ha:  NOT  POISSON 

REJECT  Ho 

did  not  fail  often,  but  there  is  an  “outlier”  vehicle  with  10  failures.  Of  the  other  groups  in  Table  1, 
the  1988  PDM  group  has  a  relatively  low  P-value.  (The  P-value  represents  the  probability  of 
obtaining  the  data  given  the  underlying  distribution  is  a  Chi-Squared  distribution.)  With  the 
exception  of  the  1989  no-PDM  vehicles,  the  failures  per  year  per  group  may  be  assumed  to  be 
Poisson  distributed. 

The  Poisson  process  assumes  that  the  failures  per  year  are  independent.  However,  the 
type  of  failure  that  occurs  influences  the  number  of  failures  per  year.  If  a  vehicle  experiences  a 
major  failure,  the  vehicle  will  be  down  for  a  signiflcant  period  of  time;  this  vehicle  will  not  be  at 
risk  during  the  downtime  and  may  experience  fewer  failures  per  year.  The  failures  per  year,  as 
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shown  by  the  service  time  survivor  curves  in  the  bootstrap  technique  section  in  this  chapter,  are 
not  independent  because  the  service  times  for  the  LVS  are  large.  Therefore,  vehicles  that 
experience  a  failure  at  the  end  of  the  year  are  potentially  not  at  risk  for  failure  a  large  portion  of 
time  the  following  year.  The  violation  of  the  independence  assumption  will  increase  the  variability 
of  the  number  of  failures  per  year  an  LVS  experiences. 

The  data  set  is  incomplete  for  the  year  1995.  The  data  set  obtained  from  the  Marine 
Corps  includes  failures  from  January  1990  to  September  1995.  The  data  did  not  include  any 
failures  that  occurred  after  the  272  day  of  1995  or  any  vehicles  that  were  not  repaired  before  that 
date.  Since  this  can  lead  to  bias,  (failures  occur  in  1994  or  1995  but  are  not  represented  in  the 
data  set  because  they  are  not  repaired  by  September  1995)  the  failures  occurring  in  1995  have 
been  omitted  from  this  part  of  the  study. 

The  Poisson  process  also  assumes  a  constant,  or  stationary,  failure  rate  for  each  class  of 
vehicles.  The  number  of  failures  per  year  is  relatively  constant  (1620  failures  in  1990,  1547 
failures  in  1991,  1652  failures  in  1992,  1472  failures  in  1993,  1732  failures  in  1994,  and  1838 
failures  in  1995),  but  the  number  of  failures  does  decrease  in  1993  and  starts  to  increase  in  1994 
and  1995. 

One  possible  interpretation  of  the  increase  in  the  number  of  failures  per  year  in  1994  and 
1995  is  the  existence  of  a  bathtub-shaped  hazard  function.  The  hazard  function  is  defined  as  the 
“ratio  of  the  probability  density  fimction  to  the  survivor  function”  (Leemis,  1995:49).  “The 
importance  of  the  hazard  function  is  that  it  indicates  the  change  in  the  failure  rate  over  the  life  of  a 
population  of  devices”  (Kapur  and  Lamberson,  1977:12).  A  bathtub-shaped  hazard  function 
assumes  a  system  experiences  an  initial  decrease  in  the  hazard  function,  but  then  increases  with 
age  (Leemis,  1995:51).  First,  in  the  infancy  stage,  the  hazard  rate  decreases  with  time.  This  is 
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often  called  the  “bum  in”  period  (Leemis,  1995:51).  Once  the  system  survives  past  this  initial 
period,  the  system  has  a  fairly  constant  hazard  function  (Leemis,  1995:51).  However,  as  the 
system  continues  to  age,  the  hazard  function  starts  to  increase  and  these  failures  are  known  as 
“wear-out”  failures  (Leemis,  1995:51).  The  increase  in  the  number  of  failures  in  1994  and  1995 
may  be  attributed  to  an  increase  in  the  hazard  function  of  the  LVS,  and  thus  failures  be  considered 
“wear-out”  failures. 

The  increasing  failure  rate  is  not  surprising  because  the  vehicles  are  getting  older,  and 
vehicle  failure  rates  tend  to  increase  with  time.  Added  to  this,  the  85  year  vehicles  are  almost  10 
years  old  in  1995  and  will  be  scheduled  for  their  mid-life  PDM  in  1996  (Joe  Littleton,  1996). 

The  failure  rates,  shown  in  Table  5,  show  the  failures  per  year  for  no-PDM  vehicles  is 
approximately  1.2  failures  per  year  and  for  PDM  vehicles  is  approximately  .33  failures  per  year. 
The  scatter  plot  of  the  number  of  failures  per  year  per  group  is  included  in  Appendix  B.  This 
graph  shows  the  extensive  variability  of  the  number  of  failures  per  year  experienced  by  the  LVS 
for  both  no-PDM  and  PDM  vehicles.  Since  this  model  explicitly  assumes  a  constant  failure  rate,  a 
Poisson  regression  is  used  to  determine  if  the  year  of  the  failure  is  significant  variables.  Table  2 
shows  the  results  of  Poisson  regression  with  only  classification  variables. 

Note  the  year  in  Table  2  represents  the  failures  that  occur  in  the  previous  year.  The 
Poisson  regression  results  are  inconclusive  because  the  test  for  goodness  of  fit  rejects  the  null 
hypothesis  that  the  model  is  Poisson.  The  link  function  for  this  model  does  not  capture  the 
variability  of  the  number  of  failures  per  year,  thus  the  model  can  not  be  assumed  to  be  Poisson. 

No  degree  of  confidence  can  be  obtained  from  the  P-values  for  the  coefficients,  however,  there  is 
evidence  that  suggests  failures  in  1993  are  significant.  The  total  number  of  failures  decrease  in 
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Table  2 


Poisson  Regression  Results  with  only  Classification  Variables 


COEFICIENTS 

STD 

ERROR 

COEFFICIENT 
STD  ERROR 

P 

VALUE 

HYPOTHESIS 

TEST 

RESULT 

a  =  .05 

CONSTANT 

-2.00726 

0.28888 

-6.95 

0 

m 

IBB 

IBB 

Reject  Ho 

PDM 

-1.22633 

0.07128 

-17.2 

0 

n 

Reject  Ho 

Y85 

2.09137 

0.29067 

7.19 

0 

M 

IBB 

IBB 

Reject  Ho 

Y86 

1.99975 

0.28885 

6.92 

0 

B 

IBB 

Reject  Ho 

Y87 

2.22741 

0.28873 

7.71 

0 

B 

IBB 

ItIHiB 

Reject  Ho 

Y88 

2.15801 

0.28869 

7.48 

0 

B 

■M»M 

Reject  Ho 

Y91 

-0.06069 

0.03585 

-1.69 

0.9448 

Ho 

Ha 

IBB 

IBB 

Do  Not 
Reject  Ho 

Y92 

-0.11651 

0.03626 

-3.21 

0.2627 

Ho 

Ha 

P7  =  0 
Pt^O 

Do  Not 
Reject  Ho 

Y93 

-0.04103 

0.03572 

-1.15 

0.9134 

Ho 

Ha 

P8=0 

Ps  ^  0 

Do  Not 
Reject  Ho 

Y94 

-0.09445 

0.03654 

-2.58 

0.0241 

Ho 

Ha 

P9=0 

Pg  0 

Reject  Ho 

DEVIANCE  9563.69 

P-VALUE  0 

DEGREES  OF  FREEDOM  783  2 


1993,  but  the  number  of  failures  increase  in  1994  and  1995,  The  significance  of  the  1993  failures 
and  the  increase  in  the  1994  and  1995  failures  implies  a  non-stationary  process.  This  deviation 
from  the  non-stationary  assumption  will  ultimately  effect  the  point  estimates  of  the  coefficients  for 
the  independent  variables  and  the  confidence  of  the  P-values  for  the  independent  variables. 

In  order  to  use  the  variable  YEARS  SINCE  LAST  PDM,  the  failures  in  a  given  year  must 
occur  uniformly  throughout  the  year.  The  results  of  the  Chi-Squared  goodness  of  fit  test,  using 
Microsoft  Excel,  for  the  failures  occurring  uniformly  throughout  the  year  are  included  in  Table  3. 
The  graphs,  depicting  the  number  of  failures  in  a  year  separated  into  20  equal  intervals,  are 
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Table  3 


Goodness  of  Fit  Results  for  the  Uniform  Distribution 


P-VALUE 

HYPOTHESIS  TEST 

RESULT 

a  =  .05 

90 

176.43 

86.86 

P-=  0.00 

Ho  UNIFORM 

Ha!  NOT  UNIFORM 

REJECT  Ho 

91 

175.03 

227.13 

P-=  0.00 

Ho  UNIFORM 

Ha!  NOT  UNIFORM 

REJECT  Ho 

92 

181.86 

135.53 

P-=  0.00 

Ho  UNIFORM 

Ha:  NOT  UNIFORM 

REJECT  Ho 

93 

167.09 

95.59 

P-=  0.00 

Ho  UNIFORM 

Ha!  NOT  UNIFORM 

REJECT  Ho 

94 

180.49 

85.11 

P-=  0.00 

Ho  UNIFORM 

Ha:  NOT  UNIFORM 

REJECT  Ho 

95 

159.78 

986.98 

P-=  0.00 

Ho  UNIFORM 

Ha:  NOT  UNIFORM 

REJECT  Ho 

included  in  Appendix  C.  None  of  the  years  have  failures  that  are  uniformly  distributed,  and  they 
all  reject  the  null  hypothesis  that  their  underlying  distributions  are  uniform.  However,  since  the 
average  day  that  failures  occur  is  close  to  the  midpoint  of  the  year  for  every  year,  this  study  uses 
an  expected  day  of  failure  of  182.5. 

Since  the  number  of  failures  per  year  per  group  can  be  assumed  to  be  Poisson  distributed 
and  the  failures  occur  near  the  midpoint  of  the  year,  a  Poisson  regression  is  used  to  attempt  to 
predict  the  number  of  failures  per  year.  The  Poisson  regression  results  using  Statistix  4. 1  are 
included  in  Table  4.  The  hypothesis  test,  that  the  model  is  Poisson,  is  rejected  resulting  in  the 
conclusion  that  the  model  is  not  Poisson.  The  results  in  Appendix  A,  however,  conclude  the 
failures  per  year  per  group  are  Poisson.  The  rejection  of  the  null  hypothesis  suggests  the 
proposed  link  function  for  the  Poisson  regression  does  not  model  the  underlying  Poisson  process. 
This,  however,  does  not  suggest  that  the  failure  process  for  each  group  does  not  follow  a  Poisson 
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Table  4 


Results  of  the  Poisson  Regression  with  the  Number  of  Failures  per  Year 


PREDICTOR 

VARIABLES 

COEFICIENTS 

STD 

ERROR 

COEFFICIENT 
STD  ERROR 

P-VALUE 

HYPOTHESIS 

TEST 

RESULT 

a  =  .05 

CONSTANT 

-2.11325 

0.28869 

-7.32 

0.0000 

Ho:  Po  =  0 

Ha:  Po5*0 

Reject  Ho 

PDM 

-1.16268 

0.07494 

-15.52 

0.0000 

Ho:  p,=0 

Ha:  P.^O 

Reject  Ho 

Y85 

2.02281 

0.29222 

6.92 

0.0000 

Ho:  P2-O 

Ha:  p2^0 

Reject  Ho 

Y86 

1.94813 

0.28970 

6.72 

0.0000 

Ho:  P3-O 

Ha:  p3^0 

Y87 

2.19211 

0.28912 

7.58 

0.0000 

Ho:  p4  =  0 

Ha:  p4^0 

Y88 

2.14012 

0.28876 

7.41 

0.0000 

Ho:  p5  =  0 

Ha:  Ps^O 

Reject  Ho 

YRAFTPDM 

0.01760 

0.00805 

2.19 

0.0288 

Ho:  P6  =  0 

Ha:  Pe^^O 

Reject  Ho 

DEVIANCE  9571.52 

P-VALUE  0 


process.  This  contradiction  is  explained  by  the  variability  of  the  number  of  failures  per  year  not 
explained  by  the  model  perhaps  due  to  the  inability  of  the  independent  variables  to  capture  any 
substantial  increase  in  the  number  of  failures  per  year.  All  the  variables  are  statistically  significant, 
but  since  the  model  is  not  Poisson,  a  statistical  confidence  of  the  variable  coefficients  can  not  be 
determined.  The  P-values  for  the  variable  coefficients  are  only  valid  if  the  model  is  significant. 

Unlike  linear  regression,  the  magnitude  of  the  effect  of  the  variables  on  the  number  of 
failures  per  year  can  not  directly  be  obtained  from  Table  4.  The  fitted  values  of  the  Poisson 
regression  model  are  obtained  using  the  function:  e^*’**^  (Siegel,  1992:180).  The  expected 
number  of  failures  for  a  no-PDM  year  85  vehicle  that  has  been  in  service  for  ten  years  is  1 .089 
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failures.  The  expected  number  of  failures  for  a  PDM  year  85  vehicle  that  has  been  in  service  for 
five  years  after  PDM  is  .3 1 19  failure.  The  other  year  group  vehicles  of  the  same  PDM  status 
display  similar  expected  number  of  failures  per  year.  This  suggests  that  the  PDM  is  an  effective 
program  that  does  decrease  the  expected  number  of  failures  per  year. 

This  model  either  overestimates  the  failures  per  year  or  grossly  underestimates  the  failures 
per  year.  Appendix  D  includes  the  scatter  plot  of  the  residuals  and  the  actual  number  of  failures 
per  year  and  the  Wilk- Shapiro  Rankit  Plot  for  normality  of  the  residuals.  The  residual  plot  in 
Appendix  D  shows  two  distinct  groups:  The  higher  error  is  the  PDM  group  while  the  lower  is  the 
no-PDM  group. 

This  model  does  not  capture  the  variability  of  PDM  failures  per  year  or  the  no-PDM 
failures  per  year.  The  model  predicts  only  a  fraction  of  a  failure  for  the  PDM  vehicles  and 
approximately  one  failure  for  no-PDM  vehicles.  If  a  vehicle  does  not  fail  in  a  year,  it  has  a  slightly 
negative  residual  for  the  PDM  vehicles  and  a  residual  of  approximately  one  for  no-PDM  vehicles. 
However,  if  the  vehicle  exhibits  multiple  failures,  there  is  a  significantly  higher  residual.  This  is 
clearly  seen  in  the  residual  plots  with  the  fitted  values.  Appendix  E  includes  the  residual  plots  for 
the  independent  variables  and  the  fitted  variables.  Figure  1  shows  the  a  plot  of  the  number  of 
failures  per  year  by  year.  Figure  1,  like  Appendix  B,  shows  the  variability  of  the  failures  per  year 
for  the  LVS.  Appendix  D  shows  an  increasing  error  rate  for  the  Poisson  regression  model  as  the 
number  of  actual  failures  increase.  This  is  explained  by  the  Poisson  regression’s  expected  number 
of  failures  per  year  for  all  the  groups  being  less  than  1 .3  failures  per  year. 

The  residual  plot  for  the  YEARS  SINCE  LAST  PDM  in  Appendix  E  shows  that  this 
variable  has  little  explanatory  power.  The  residuals  show  no  distinct  pattern.  With  the  lack  of  a 
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Scatter  Plot  of  NUMFAIL  vs  YEAR 


0  H  • 


YEAR 


Figure  1  Plot  of  the  Failures  per  Year  by  Year 

significant  variable  that  captures  usage,  the  increase  in  the  amount  of  failures  can  not  be  explained 
in  this  model. 

The  Wilk-Shapiro  Rankit  plot  in  Appendix  D  shows  that  the  error  terms  are  not  normally 
distributed.  The  error  terms  for  the  Poisson  regression  model  are  assumed  to  be  normally 
distributed,  but  in  this  case  are  not  because  the  large  variability  in  the  number  of  failures  per  year 
that  is  not  captured  by  this  model..  Nelder  and  McCullagh  offer  a  variance  stabilizing  transform 
for  the  Poisson  distribution: 

E(Y''^)  « 

VAR(*^^)  » ’74 

(Nelder  and  McCullagh,  1992;  196) 
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However,  this  transformation  only  applies  to  large  values  of  p  (Nelder  and  McCullagh, 

1992;  196).  This  transformation  can  not  be  used  in  this  case  because  the  expected  number  of 
failures  for  a  PDM  vehicle  is  less  than  one.  There  is  evidence  that  the  PDM  program  does  reduce 
failures,  but  Poisson  regression  provides  no  level  of  statistical  confidence  for  the  independent 
variable  point  estimates  because  the  data  do  not  support  the  model  assumptions. 

To  test  whether  the  failures  per  year  are  significantly  different  for  PDM  and  no-PDM 
vehicles,  an  F-test  was  performed  on  each  year  group’s  failure  rates.  Table  5  shows  the  results  of 
the  tests. 


Table  5 

F-test  for  the  Difference  in  Failure  Rates  for  PDM  and  no-PDM  Vehicles 


CLASS 

NUMBER 

OF 

FAILURES 

H 

STD 

DEV 

■ 

k 

p 

VALUE 

RESULT 

85  no-PDM 

617.04 

1.0115 

1.1605 

x\ 

85  PDM 

34.575 

0.3602 

0.754 

il 

mom 

2.8081 

P=0 

REJECT  Ho 

86  no-PDM 

i\ 

Ho 

-^2 

■■■ 

■■■I 

86  PDM 

ii 

Ha 

3.5680 

87  no-PDM 

2343.1 

1.1716 

Ho 

A-i  =  ^2 

m 

87  PDM 

0.3343 

3.5046 

REJECT  Ho 

88  no-PDM 

88  PDM 

27.944 

0.3408 

0.8463 

X-2 

P=0 

REJECT  Ho 

89  no-PDWP 

12 

0.1263 

Table  5  shows  the  failure  rates  for  no-PDM  vehicles  are  approximately  one  failure  per  year,  and 
the  failure  rates  for  PDM  vehicles  are  approximately  one  third  failure  per  year.  This  is 
approximately  the  same  results  of  the  Poisson  regression.  The  hypothesis  tests  reject  Ho,  which 
implies  the  failure  rates  for  PDM  and  no-PDM  are  significantly  different,  and  PDM  significantly 
reduces  the  number  of  failures  per  year. 
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Boot  Strap  Technique 


Bootstrap  estimates  for  number  of  failures  per  year  and  downtime  per  year  are  not 
obtained  for  the  1989  year  group  vehicles  because  of  the  relatively  small  sample  size.  The 
bootstrap  technique  uses  all  the  times  to  failure  and  the  service  time  distributions  from  all  the 
years  in  the  data  set.  It  has  been  shown  that  the  failure  rates  are  increasing  in  1994  and  1995. 
Thus,  when  the  bootstrap  samples  from  the  empirical  time  to  failure  distribution  prior  to  1993,  the 
expected  time  to  failure  will  be  larger  than  the  current  (1995)  time  to  failure.  This  potentially 
results  in  a  decrease  in  the  number  of  failures  per  year.  This  will  bring  bias  to  the  bootstrap  result, 
but  the  ultimate  result  will  be  a  more  conservative  estimate. 

The  results  of  the  hypothesis  testing  on  the  Kaplan-Meier  estimation  of  the  survivor 
curves  for  PDM  and  no-PDM  using  S-plus  are  included  in  Appendix  F.  For  all  the  year  groups, 
the  failure  curves  for  PDM  and  no-PDM  are  significantly  different.  This  suggests  the  PDM 
vehicles  survive  longer  without  failure.  The  service  time  curves  for  PDM  and  no-PDM,  with  the 
exception  of  the  1988  year  group,  are  significantly  different.  This  suggests  that  the  PDM  vehicles 
(except  the  1988  year  group)  are  not  out  of  service  as  long  as  no-PDM  vehicles.  The  plots  for 
the  survivor  curves  are  included  in  Appendix  G.  The  results  of  the  bootstrap  technique  are 
included  in  Table  6,  the  95  percent  confidence  intervals  are  included  in  Appendix  H  and  graphs  of 
the  bootstrap  results  are  included  in  Appendix  I.  While  the  number  of  failures  per  year  developed 
from  the  Poisson  regression  could  not  pass  the  rigors  of  a  statistical  test,  they  are  not  drastically 
different  from  the  significant  values  obtained  using  the  bootstrap  technique. 

The  bootstrap  estimates  of  dovmtime  for  no-PDM  and  PDM  are  distinctly  different.  The 
no-PDM  group  is  out  of  service  3  to  4  times  as  long  as  the  PDM  group.  The  significant  amount 
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Table  6 


Bootstrap  Technique  Results 


GROUP 

DOWN  TIME 

STD  DEV 

NUM  FAIL 

STD  DEV 

DOWNTIME 
NEXT  YEAR 

STD  DEV 

85  NO  PDM 

119.601665 

3.4900039 

1.1495375 

0.027741 

56.3517945 

3.245101 

85  PDM 

41.186188 

2.4101167 

0.561372 

0.023662 

13.694968 

1.569306 

86  NO  PDM 

115.434678 

3.5569385 

1.1736125 

0.02941 

59.1211875 

3.508131 

86  PDM 

18.1646545 

1.7518671 

0.3586975 

0.020442 

4.075965 

0.934783 

87  NO  PDM 

135.033182 

3.9347049 

1 .430694 

0.02987 

70.3846965 

3.578988 

87  PDM 

52.8839895 

2.8621704 

0.619728 

0.026985 

13.964259 

1.736707 

88  NO  PDM 

112.994979 

3.3361717 

1.333514 

0.032129 

52.6088685 

3.25806 

88  PDM 

32.161272 

2.4872614 

0.351101 

0.019485" 

18.2325805 

2.154502 

of  time  the  LVS  stays  out  of  service  supports  the  conclusion  that  the  number  of  failures  per  year 
is  not  independent  from  year  to  year  because  if  a  failure  occurs  near  the  end  of  a  year,  the  vehicle 
will  not  be  at  risk  to  fail  a  large  portion  of  the  following  year.  When  the  no-PDM  vehicles  are  out 
of  service  for  approximately  one  third  of  the  year,  they  are  not  at  risk  to  fail  as  often  as  PDM 
vehicles.  The  no-PDM  vehicles  fail  more  than  twice  as  often,  while  the  PDM  vehicles  are  at  risk 
of  failure  approximately  90  days  a  year  longer  than  the  no-PDM  vehicles. 

The  results  of  this  research  suggests  the  number  of  failures  per  year  is  approximately  one 
failure  per  year  for  no-PDM  vehicles  and  is  approximately  one  third  of  a  failure  per  year  for  PDM 
vehicles,  and  the  no-PDM  vehicles  are  unserviceable  for  a  significant  period  of  time  after  failure. 
This  has  many  possible  implications:  the  vehicles  can  be  experiencing  major  failures,  and  the 
vehicle  is  essentially  rebuilt  at  base  level,  or  the  supply  system  could  have  difficulty  re-supplying 
the  maintenance  facilities  with  the  needed  repair  parts.  Of  course,  there  may  be  other  factors 
beyond  the  reach  of  this  analysis.  Research  does  imply,  however,  that  there  is  a  significant 
decrease  in  the  number  of  failures  per  year  and  the  downtime  per  year  for  PDM  vehicles. 
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The  results  of  the  Poisson  regression  reveal  that  there  may  be  a  difference  in  the  number 
of  failures  per  year  for  PDM  and  no-PDM  vehicles,  but  a  statistical  confidence  of  the  coefficients 
can  not  be  obtained  because  of  the  lack  of  model  significance.  However,  the  expected  number  of 
failures  per  year  for  each  group  is  approximately  the  same  as  those  obtained  from  the  bootstrap 
estimation.  The  1994  and  1995  failure  data  indicate  that  the  failure  rates  for  the  LVS  vehicles  are 
increasing.  Therefore,  the  bootstrap  results  represent  a  conservative  estimate,  and  the  availability 
of  the  LVS  in  the  future  may  be  worse. 

The  bootstrap  estimates  show  that  the  LVS  is  out  of  service  approximately  33  percent  of 
the  time  for  no-PDM  vehicles.  However,  sending  the  vehicles  to  PDM  will  decrease  this  out 
service  rate  to  approximately  10  percent.  This  study  also  reveals  a  significant  problem  bringing  a 
broken  no-PDM  LVS  back  into  service.  While  theoretical  statistical  questions  remain  due  to  the 
data  limitations,  PDM  appears  to  significantly  reduce  the  number  of  failures  per  year.  The 
amount  of  time  an  LVS  remains  in  the  maintenance  facility  is  a  major  factor  for  the  decreased 
availability  of  the  LVS,  particularly  for  no-PDM  vehicles. 
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V.  Conclusions  and  Recommendations 


The  purpose  of  this  chapter  is  to  present  the  conclusions  and  recommendations  obtained  in 
this  research  study.  Initially,  the  research  questions  will  be  addressed;  then  the  limitations  of  the 
study  will  be  presented.  Finally,  the  results  and  implications  will  be  discussed  followed  by 
recommendations  for  further  study. 

Research  Questions 

The  research  questions  answered  in  this  study  are: 

1 .  Does  PDM  reduce  the  number  of  failures  per  year? 

2.  Does  PDM  reduce  the  amount  of  downtime  once  the  weapon  system  has 
failed? 

The  results  presented  in  Chapter  4  show  that  PDM  appears  to  reduce  the  number  of 
failures  per  year  from  approximately  1 .3  failures  per  year  to  less  than  one  half  of  a  failure  per  year. 
Although  no  confidence  can  be  obtained  from  the  Poisson  regression,  the  model  does  suggest  that 
PDM  does  reduce  the  number  of  failures  per  year.  The  hypothesis  testing  on  the  failure  rates 
within  each  vehicle  year  group  concludes  PDM  vehicle  failure  rates  are  significantly  lower  than 
no-PDM  vehicle  failure  rates.  The  bootstrap  technique  further  illustrates  the  reduction  of  failures 
per  year  for  PDM  vehicles. 

The  failure  rate  estimates  for  PDM  and  no-PDM  vehicles  obtained  from  all  of  the 
techniques  used  in  Chapter  4  are  consistent  for  each  vehicle  year  group.  Since  both  parametric 
and  non-parametric  techniques  are  used  to  obtain  the  failure  rates,  the  consistency  between  the 
failure  rates  suggest  that  the  failure  rate  estimates  are  close  to  the  true  failure  rates. 
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The  survivor  curves  for  time  to  failure  show  significant  differences  between  PDM  and  no- 
PDM  vehicles.  The  PDM  vehicles  have  a  longer  time  between  failures  than  the  no-PDM  vehicles. 
The  survivor  curves  for  the  service  time,  except  for  the  1988  vehicles,  are  also  significantly 
different  for  PDM  vehicles  and  no-PDM  vehicles.  This  indicates  no-PDM  vehicles  remain  in  the 
maintenance  facility  materially  longer  than  PDM  vehicles. 

The  bootstrap  estimates  for  downtime  further  validate  the  conclusions  obtained  from  the 
survivor  curve  analysis.  The  bootstrap  estimates  reveal  the  magnitude  of  the  differences  between 
PDM  vehicles  and  no-PDM  vehicle  downtime  per  year.  No-PDM  vehicles  remain  unserviceable 
for  approximately  3.5  times  longer  than  PDM  vehicles  and  are  down  approximately  120  days  a 
year. 

T.imitations  of  the  Study 

The  Poisson  regression  model  rejects  the  hypothesis  that  the  model  is  Poisson.  The  model 
rejects  this  hypothesis  because  of  the  unexplained  variability  of  the  number  of  failures  per  year. 
Since  YEARS  SINCE  LAST  PDM  has  a  small  effect  estimate,  this  variable  does  not  significantly 
increase  the  number  of  failures  per  year  and  does  not  adequately  capture  usage.  To  reduce  the 
unexplained  variability  and  to  obtain  better  effect  estimates,  a  variable,  or  variables,  must  be 
introduced  into  this  model  that  more  accurately  depict  usage.  Variables  such  as  miles  per  year, 
hours  of  use  per  year,  or  type  of  mission  of  the  vehicle  might  better  explain  the  changing  number 
of  failures  per  year  than  YEARS  SINCE  LAST  PDM. 

The  survivor  curve  estimates  of  the  time  to  failure  implicitly  assume  that  the  last  failure 
before  the  date  of  request  for  PDM  is  a  catastrophic  failure,  and  the  vehicle  is  immediately 
shipped  to  PDM.  This  implies  no  right  censored  data  for  vehicles  that  went  through  PDM.  In  the 
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case  that  a  vehicle  is  working  at  the  time  of  PDM,  the  amount  of  time  the  vehicle  remained  in 
service  from  its  last  failure  to  the  date  the  vehicle  was  shipped  to  the  depot  should  be  considered  a 
right  censored  data  point.  Instead,  this  time  is  added  to  the  time  for  the  first  time  to  failure  after 
PDM.  This  assumption  will  potentially  decrease  the  time  to  failure  survivor  curve  estimates  for 
the  no-PDM  vehicles  and  increase  the  bootstrap  estimates  for  failures  per  year. 

In  addition,  the  date  of  completion  of  the  PDM  is  not  available,  so  the  time  for  the  first 
failure  after  PDM  also  includes;  shipping  time  to  the  depot,  time  in  the  depot,  and  shipping  time 
back  to  the  unit,  as  well  as  the  out-of-service  time.  This  over-estimation  of  the  first  time  to  failure 
after  PDM  will  potentially  increase  the  PDM  time  to  failure  survivor  curve  estimates  and  decrease 
the  bootstrap  estimates  for  failures  per  year.  However,  since  the  failures  per  year  are  consistent 
among  all  the  techniques  used  in  this  study,  apparently  these  assumptions  did  not  dramatically 
effect  the  bootstrap  estimates. 

All  the  models  in  Chapter  4  assume  a  stationary  failure  rate,  but  the  increase  in  the  number 
of  failures  per  year  in  1994  and  1995  indicates  the  failure  rates  are  rising.  Because  the  number  of 
failures  per  year  is  increasing  with  time,  the  effect  estimates  for  the  number  of  failures  per  year  are 
understated.  In  order  to  obtain  better  effect  estimates,  failure  data  after  1993  should  be  analyzed 
separately  because  this  is  the  point  in  time  where  the  failure  rate  starts  increasing. 

Results  and  Implications 

PDM  significantly  reduces  the  number  of  failures  per  year  and  substantially  reduces  the 
downtime  per  year.  However,  the  results  in  Chapter  4  show  relatively  small  failure  rates  for  both 
PDM  and  no-PDM  vehicles  with  high  out  of  service  times.  As  stated  in  Chapter  4,  there  are 
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several  possible  causes.  The  vehicles  may  be  experiencing  critical  or  catastrophic  failures  and  are 
rebuilt  at  base  level,  or  the  supply  system  may  not  adequately  re-supply  the  maintenance  facilities. 

Either  scenario  has  direct  implications  to  PDM.  If  the  weapon  system  experiences  a 
critical  or  catastrophic  failure  which  requires  PDM,  the  base  level  maintenance  is  not  properly 
equipped  to  attempt  the  rebuild.  The  repair  will  occur  either  at  base  level  or  at  the  depot,  but  it 
has  been  shown  that  PDM  is  far  preferable  to  a  base  level  overhaul  because  PDM  significantly 
reduces  the  number  of  failures  per  year. 

There  is  evidence  that  indicates  the  supply  system  is  experiencing  difficulties  in  supporting 
the  TVS.  The  Marine  Corps  LVS  item  manager  stated  that  there  existed  a  supply  problem  with 
the  LVS,  and  many  parts  have  a  large  lead  time  between  a  part  request  and  delivery.  If  the 
current  supply  system  can  not  adequately  re-supply  the  maintenance  facilities,  reducing  the 
number  of  failures  per  year  with  the  PDM  program  will  significantly  increase  weapon  system 
availability.  Since  these  vehicles  are  aging  and  nearing  their  mid-life  rebuild,  these  vehicles  will 
inevitably  experience  more  critical  or  catastrophic  failures  per  year.  With  any  increase  in  these 
t5q)es  of  failures  per  year,  the  projected  effect  on  the  supply  system  will  only  further  reduce  the 
availability  of  the  LVS  and  make  PDM  even  more  critical. 

The  cost  for  PDM,  regardless  of  the  actual  maintenance  performed  on  a  weapon  system,  is 
considered  a  fixed  cost.  The  PDM  cost  for  the  LVS  is  $72,157  for  the  depot  located  Albany, 
Georgia  and  $75,494  for  the  depot  located  in  Barstow,  California  (Lydia  Burch,  1996).  The 
current  price  for  procurement  of  a  new  LVS  is  approximately  $172,000  plus  $9,000  for  additional 
spare  parts.  Thus,  the  total  cost  of  a  new  LVS  is  approximately  $181,000.  These  prices  are 
based  on  the  procurement  of  42  to  45  LVSs  (Joe  Littleton,  1996).  The  increase  in  availability,  if 
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PDM  is  performed,  is  approximately  24.7  percent  J .  The  downtime  for  a  no-PDM 

vehicle  is  approximately  120  days  and  the  downtime  for  a  PDM  vehicle  is  approximately  30  days. 
If  200  vehicles  are  scheduled  for  PDM,  but  are  not  funded,  approximately  54  vehicles  need  to  be 


procured  to  obtain  the  same  availability 


200(.247) 
.918  . 


Note,  the  availability  of  a  PDM  vehicle  is 


91.8  percent,  and  PDM  vehicles  exhibit  failure  rates  similar  to  new  vehicles.  The  cost  for 
procuring  54  vehicles  is  approximately  $9.8  million,  and  the  cost  for  the  PDM  is  approximately 
$14.8  million. 

However,  this  analysis  does  not  take  into  account  the  cost  of  maintenance  at  the  base 
level.  Furthermore,  adding  54  vehicles  to  the  fleet  only  temporarily  maintains  availability  and 
masks  the  real  issue.  This  solution  increases  the  demands  placed  on  a  supply  system  that  currently 
can  not  support  the  fleet,  and  the  200  no-PDM  vehicles  will  continue  to  deteriorate.  In  addition, 
this  analysis  assumes  that  the  availability  of  the  no-PDM  vehicles  remains  constant  the  next  year, 
but  this  research  has  shown  that  the  failure  rate  is  increasing. 

As  more  no-PDM  LVS’s  experience  critical  or  catastrophic  failures  and  as  the  demand  for 
replacement  parts  increase,  the  availability  of  the  no-PDM  LVS  will  inevitable  decrease  further. 
When  the  availability  of  the  no-PDM  LVS  decreases  from  67. 1  to  54.2  percent,  the  procurement 
costs  and  PDM  costs  will  yield  the  same  availability.  Procuring  replacement  vehicles  until  the 
availability  rate  reaches  54.2  percent  is  not  cost  effective. 
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Recommendations  for  Follow-on  Research 


This  research  study  is  limited  by  the  assumptions  made  about  the  data  and  the  assumptions 
of  a  Poisson  process.  Although  the  results  conclude  that  PDM  is  significant  in  reducing  the 
number  of  failures  per  year  and  increasing  the  availability  of  the  LVS,  these  estimates  for  number 
of  failures  per  year  and  downtime  per  year  are  conservative. 

More  attention  to  detail  should  be  used  in  the  data  collection  process.  First,  the  Marine 
Corps  depot  needs  an  automated  data  base  to  provide  an  audit  trail  for  a  weapon  system  through 
the  depot.  Data  such  as  the  date  a  weapon  system  is  received  in  the  depot  and  the  date  a  weapon 
system  is  shipped  back  to  the  unit  should  be  included  in  the  data  base.  A  maintenance  history 
should  show  the  depot  maintenance  performed  on  the  weapon  system.  These  data  can  support 
analysis  to  determine  if  maintenance  in  the  field  had  to  be  re-accomplished  at  the  depot.  The 
manufacturing  date  of  the  LVS  is  imperative  in  determining  the  age  of  the  system  and  the  Marine 
Corps  currently  does  not  track  this  date. 

In  order  to  obtain  better  effect  estimates  for  the  Poisson  regression,  a  variable  such  as 
mileage  per  year  or  hours  of  use  per  year  should  capture  usage.  Currently,  data  are  only  collected 
upon  failure.  A  model  such  as  the  one  proposed  in  Chapter  3  requires  mileage  or  usage  per  year. 
A  variable  such  as  mission  tjrpe  or  location  would  also  help  explain  the  number  of  failures  per  year 
because  there  are  a  variety  of  missions  in  the  Marine  Corps. 

Finally,  there  is  evidence  that  suggests  the  number  of  failures  per  year  is  increasing  in  1994 
and  1995.  A  follow  on  study  should  focus  on  failure  data  after  1993.  The  effect  estimates  would 
better  represent  the  current  status  of  the  weapon  system  and  would  not  be  biased  by  the  lower 
failure  rates  experienced  in  1990-1993. 
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APPENDIX  A:  Graphs  of  Poisson  Distributions 


Comparison  of  Yr  85  no-PDM  Failure  Distribution  and  Poisson{1.01) 


Comparison  ofYr  85  PDM  Failure  Distribution  and  Poisson(0.28) 


Input 


Poisson 
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Comparison  of  Yr86  no-PDM  Failure  Distribution  and  Poisson{0.97) 


Comparison  of  Yr  86  PDM  Failure  Distribution  and  Poisson(0,26) 


APPENDIX  B:  Scatter  Plot  of  the  Number  of  Failures  per  Year  per  Group 


GROUP- YEAR 


Group  1  is  1985  year  vehicles  with  no-PDM. 

Group  2  is  1985  year  vehicles  with  PDM. 

Group  3  is  1986  year  vehicles  with  no-PDM. 

Group  4  is  1986  year  vehicles  with  PDM. 

Group  5  is  1987  year  vehicles  with  no-PDM. 

Group  6  is  1987  year  vehicles  with  PDM. 

Group  7  is  1988  year  vehicles  with  no-PDM. 

Group  8  is  1988  year  vehicles  with  PDM. 

Group  9  is  1989  year  vehicles  with  no-PDM. 

The  first  column  represents  the  failures  occurring  in  the  year  1990. 
The  second  column  represents  the  failures  occurring  in  the  year  1991 
The  third  column  represents  the  failures  occurring  in  the  year  1992. 
The  forth  column  represents  the  failures  occurring  in  the  year  1993. 
The  fifth  column  represents  the  failures  occurring  in  the  year  1994. 
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Ordered  Data 


APPENDIX  D:  Scatter  Plot  of  the  Residuals  with  Number  of  Failures  per  Year 

and  the  Wilk-Shipiro  Rankit  Plot 


Scatter  Plot  of  RBS  vs  NUMFAIL 


Wilk— Shapiro  /  Rankit  Plot  of  RES 
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Scatter  Plot  of  RES  vs  GROUP 
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Appendix  F:  Results  of  the  Hypothesis  Testing  for  PDM  and  no-PDM  Kaplan- 

Meier  Survivor  Curves 


Year  85  vehicles  time  to  failure  curves 


Log  rank  test 


I _ 

Expected 

UJ 

< 

UJ 

1 

O 

554 

504.76 

4.803 

PDM 

66 

38 

87.24 

27.789 

Chisq=34.5 

p=0.000000004206 

Result:  Reject  Ho  with  a  =  .05 

Peto-Peto  modification 


N 

Observed 

Expected 

(0-E)''2/E 

NOPDM 

743 

330.11 

307.61 

1.645 

PDM 

66 

17.99 

40.49 

12.499 

Chisq=22.6 

p=0.000001968 

Result:  Reject  Ho  with  a  =  .05 

Year  86  vehicles  time  to  failure  curves 


Log  rank  test 


N 

Observed 

Expected 

(0-E)'^2/E 

NOPDM 

2465 

1741 

1588.4 

14.66 

PDM 

197 

98 

250.6 

92.92 

Chisq=113.6 

p=0 

Result:  Reject  Ho  with  a  =  .05 
Peto-Peto  modification 


N 

Observed 

Expected 

(0-Ey2/E 

NOPDM 

2465 

1093.73 

1013.9 

6.289 

PDM 

197 

45.98 

125.8 

50.67 

Chisq=91.8 

p=0 

Result:  Reject  Ho  with  a  =  .05 
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Year  87  vehicles  time  to  failure  curves 


Log  rank  test 


N 

Observed 

Expected 

(0-E)^2/E 

NOPDM 

2261 

2168.8 

3.92 

PDM 

■0^ 

64 

156.2 

54.43 

Chisq=60.9 

p=0.000000000000005884 
Result:  Reject  Ho  with  a  =  .05 

Peto-Peto  modification 


N 

Observed 

Expected 

(0-E)''2/E| 

NOPDM 

2956 

1329.4 

1293.02 

PDM 

115 

33.1 

69.51 

19.075] 

Chisq=32.7 

p=0.00000001072 

Result:  Reject  Ho  with  a  =  .05 


Year  88  vehicles  time  to  failure  curves 


Log  rank  test 


N 

Observed 

Expected 

(0-E)^2/e] 

NOPDM 

3231 

2318.14 

PDM 

49 

23 

Chisq=28.2 

p=0.0000001114 

Result:  Reject  Ho  with  a  =  .05 

Peto-Peto  modification 


N 

Observed 

Expected 

(0-E)''2/E 

NOPDM 

3231 

1438.143 

0.3403 

PDM  ^ 

49 

9.905 

31.86 

15.1275 

Chisq=25.1 

p=0.0000005389 

Result:  Reject  Ho  with  a  =  .05 
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Year  85  vehicles  service  time  curves 


Log  rank  test 


N 

Observed 

Expected 

(0-E)''2/E 

NOPDM 

688 

688 

701.81 

0.2717 

PDM 

39 

39 

25.19 

7.5697 

Chisq=7.9 

p=0.004813 

Result:  Reject  Ho  with  a  =  .05 
Peto-Peto  modification 


N 

Observed 

Expected 

(0-E)''2/E 

NOPDM 

688 

341.49 

350.81 

0.2473 

PDM 

39 

24.29 

14.98 

5.7919 

Chisq=8.6 

p=0.003445 

Result;  Reject  Ho  with  a  =  .05 

Year  86  vehicles  service  time  curves 


Log  rank  test 


N 

Observed 

Expected 

(0-E)''2/E  1 

NOPDM 

2206 

PDM 

108 

108 

50.92 

Chisq=66.4 

p=0.0000000000000003331 
Result:  Reject  Ho  with  a  =  .05 

Peto-Peto  modification 


N 

Observed 

Expected 

(0-E)''2/^ 

NOPDM 

2206 

1086.13 

1131.88 

PDM 

108 

77.36 

31.61 

Chisq=92.5 

p=0 

Result:  Reject  Ho  with  a  =  .05 
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Year  87  vehicles  service  time  curves 


Log  rank  test 


N 

Observed 

Expected 

(0-E)'‘2/En 

NOPDM 

2679 

2679 

PDM 

72 

72 

56.4 

4.31431 

Chisq=4.4 

p=0.03491 

Result:  Reject  Ho  with  a  =  .05 
Peto-Peto  modification 


N 

Observed 

Expected 

(0-E)'‘2/E 

NOPDM 

2679 

1342.35 

1351.67 

0.06424 

PDM 

72 

40.86 

Chisq=4 

p=0.04438 

Result:  Reject  Ho  with  a  =  .05 


Year  88  vehicles  service  time  curves 


Log  rank  test 


N 

Observed 

Expected 

(0-E)'‘2/E  1 

NOPDM 

2829 

2829 

PDM 

26 

26 

Chisq=.01 

p=0.795 

Result:  Do  not  Reject  Ho  with  a  =  .05 
Peto-Peto  modification 


N 

Observed 

Expected 

(0-E)''2/E 

NOPDM 

2829 

1422.7 

1425.02 

0.003792 

PDM 

26 

14.27 

11.94 

0.452543 

Chisq=0.7 

p=0.4013 

Result:  Do  not  Reject  Ho  with  a  =  .05 
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APPNDIX  G:  The  Kaplan-Meier  Survivor  Curves 
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APPENDIX  H:  95  Percent  Confidence  Intervals  on  Bootstrap  Results 


Confidence  Intervals  for  Downtime  Current  Year 


GROUP 

LOWER  Bound 

DOWNTIME 

UPPER  BOUND 

85  NO  PDM 

113.8611 

X 

125.3422 

85  PDM 

37.2219 

X 

45.15048 

86  NO  PDM 

109.584 

X 

121.2853 

86  PDM 

15.28309 

X 

21.04622 

87  NO  PDM 

128.5612 

X 

141.5052 

87  PDM 

48.17614 

X 

57.59184 

88  NO  PDM 

107.5075 

X 

118.4825 

88  PDM 

28.07009 

X 

36.25245 

Confidence  Intervals  for  Number  of  Failures  per  Year 


GROUP 

LOWER  BOUND 

FAILURE  RATE 

UPPER  BOUND 

85  NO  PDM 

1.103908 

X 

1.195167 

85  PDM 

0.522451 

X 

0.600293 

86  NO  PDM 

1.125237 

X 

1.221988 

86  PDM 

0.325073 

X 

0.392322 

87  NO  PDM 

1.381562 

X 

1.479826 

87  PDM 

0.575342 

X 

0.664114 

88  NO  PDM 

1.280667 

X 

1.386361 

88  PDM 

0.319051 

X 

0.383151 

Confidence  Intervals  for  Downtime  the  Following  Year 


GROUP 

LOWER  BOUND 

DOWNTIME 

UPPER  BOUND 

85  NO  PDM 

51.01408 

61.68951 

85  PDM 

11.11369 

X 

16.27625 

86  NO  PDM 

53.35083 

X 

64.89155 

86  PDM 

2.538384 

X 

5.613546 

87  NO  PDI^ 

64.49779 

X 

76.27161 

87  PDM 

11.10763  1 

X 

16.82089 

88  NO  PDM 

47.24984 

X 

57.9679 

88  PDM 

14.68874 

X 

21.77642 
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APPENDIX  I:  Graphs  of  Bootstrap  Results 


BOOTSTRAP  RESULTS  FOR  DOWNTIME 


Y85  Y86  Y87  Y88 


BOOTSTRAP  RESULTS  FOR  FAILURES  PER  YEAR 


83 


BTBIJOGRAPHY 


Adams,  Mark.  “Depot  Maintenance  Float  Allowance  Analysis.”  Report  to  HQMC/LPM. 
November  1993. 

Bachmann,  M.C.  Briefing  handouts,  Joint  Depot  Maintenance  Analysis  Group  Briefing. 
Gentile  Air  Force  Station,  Dayton  OH,  28  October  1994. 

Bachmann,  M.C.  Briefing  handouts,  JG-LR  DM  Backlog  Assessment  Subgroup  Briefing. 
Gentile  Air  Force  Station,  Dayton  OH,  September  1994. 

Bachmann,  M.C.  Briefing  handouts,  JG-LR  DM  Backlog  Assessment  Subgroup  Briefing. 
Gentile  Air  Force  Station,  Dayton  OH,  20  April  1995. 

Bachmann,  M.C.  Briefing  handouts.  Depot  Maintenance  Backlog  Assessment  Subgroup 
Briefing  for  Joint  Logistics  Commanders.  1994. 

Bradley  Vehicle  Study.  National  Training  Center,  Ft.  Erwin  CA. 

Brill,  John  R.  Preventive  Maintenance  Intervals  for  Components  of  the  F-15/F100 

Aircraft  Engine.  MS  Thesis,  LSSR  93-83.  School  of  Systems  and  Logistics,  Air 
Force  Institute  of  Technology  (AU),  Wright  Patterson  AFB  OH,  September  1983 
(AD-Al  35637). 

Burch,  Lydia.  Telephone  Interview.  22  February  1996. 

Byrd,  R.R..  General  Officer  Symposium  1994  Issue  Paper:  Readiness-Predictive.  14 
September  1994. 

Chung,  David.  Economic  Analysis  for  the  Ml  Tank  Inspect  and  Repair  Only  as  Necessary 
Program.  Report  No.  DN-L-52494.  Aberdeen  Proving  Ground  MD;  US  Army 
Materiel  Systems  Analysis  Activity,  May  1990. 

Chung,  David  and  Clarke  J.  Fox.  Reliability-Centered  Inspect  and  Repair  Only  as 

Necessary  IRC-IRONI  Test  Program.  Technical  Report  No.  494.  Aberdeen 
Proving  Ground  MD;  US  Army  Materiel  Systems  Analysis  Activity,  January  1991 
(AD-B157417L). 

Congressional  Budget  Office.  Trends  in  Selected  Indicators  of  Military  Readiness,  1980 
Through  1993.  Washington  DC,  March  1994. 

Crawford,  Patti.  Telephone  Interview.  5  November  1995. 


84 


Curtin,  Neil  P.  Testimony  on  Military  Readiness:  Current  Indicators  Need  to  be 

Expanded  for  a  More  Comprehensive  Assessment.  Washington  DC:  General 
Accounting  Office,  21  April  1994  (GAO/T-NSIAD-949160). 

Dienemann,  Paul  F.  Estimating  Cost  Uncertainty  Using  Monte  Carlo  Techniques. 

Contract  AF  49(638)  1700.  Santa  Monica  CA:  The  RAND  Corporation,  January  1966. 

A  Discussion  of  Modeling  Support  for  PPM  (PPM)  Analysis 

and  Assessments.  Synergy  Inc.,  Washington  DC,  30  August  1994  (GAO/T- 
NSIAD-95-117). 

Effect  of  Unfunded  Deferred  Requirements  (UDR)  for  Depot  Maintenance  on  Readiness. 
Unpublished  Report  to  JDMAG.  Gentile  Air  Force  Station,  Dayton  OH,  1994. 

Efron,  Bradley,  and  Robert  J.  Tibshirani.  An  Introduction  to  the  Bootstrap.  New  York; 
Chapman  &  Hall,  1993. 

Gebicke,  Mark  E.  Testimony  on  Military  Readiness:  Improved  Assessment  Measures  are 
Evolving.  Washington  DC:  General  Accounting  Office,  16  March  1995. 

General  Accounting  Office.  Military  Readiness:  POD  Needs  to  Develop  a  More 
Comprehensive  Measurement  System.  Washington  DC,  October  1994 
(GAO/NSIAD-95-29). 

Henderson,  Frank  F.  Briefing  handouts.  Joint  Group  on  Logistics  Readiness  Briefing, 

Joint  Logistics  Commanders  Conference.  9  June  1994. 

Holzer,  Robert.  “Pentagon  Chiefs  Get  New  Policy  Weapon,”  Defense  News,  January  30- 
February  5  1995. 

Joint  Logistic  Commanders.  Paper  to  the  Secretary  of  Defense.  “JLC  Study  of  Depot 
Maintenance  Investment  Strategy.”  October  6  1988. 

Kalbfleisch,  John  D  ,  and  Ross  L.  Prentice.  The  Statistical  Analysis  of  Failure  Time  Data.  New 
York:  John  Wiley  and  Sons,  1980. 

Kapur,  K.  C.,  and  L.  R.  Lamberson.  Reliability  in  Engineering  Design.  New  York;  John  Wiley 
and  Sons,  1977. 

Keller,  A.Z.,  S.  J-Fendi  Al-Saadi,  and  L.  Leckie.  “Reliability  Assessment  of  Fire-Fighting 
Vehicles  and  Equipment,”  International  Journal  of  Quality  and  ReliabilitY 
Management.  9:  42-51  (1992). 

Kephart,  Steve  D.,  and  Richard  C.  Roberts.  “Aligning  Demand  for  Spare  Parts  with  their 


85 


Underlying  Failure  Models.”  MS  Thesis,  LAL  95S-9.  School  of  Logistics  and  Acquisition 
Management,  Air  Force  Institute  of  Technology  (AU),  Wright  Patterson  AFB  OH, 
September  1995. 

Klapper,  Larry  S.  “The  Readiness  Impacts  of  Depot  Maintenance  Unfunded  Deferred 

Requirements.”  Briefing  to  Marine  Corps  Logistics  Base  Albany  GA,  14  April  1994. 

Leemis,  Lawrence  M.  Reliability  Probabilistic  Models  and  Statistical  Methods.  New  Jersey: 
Prentice  Hall  Inc.,  1995. 

Law,  Averill  M.,  and  David  W.  Kelton.  Simulation  Modeling  &  Analysis.  2”^  ed.  New  York; 
McGraw-Hill,  Inc.,  1991. 

Littleton,  Joe.  Telephone  Interview.  16  February  1996. 

Logistics  Management  Institute.  The  Readiness  Impacts  of  Depot  Maintenance  Unfunded 
Deferred  Requirements  (UDR).  Report  for  JDMAG.  6  May  1994. 

Love,  R.E.  Reliability  Centered  Maintenance.  Newport  RI:  Naval  War  College,  1 1 
February  1991  (AD-A236  693). 

“Management  Report;  Elixirs  for  an  Aging  Power  System,”  Electrical  World.  30-33 
(March  1989). 

McCullagh,  P.,  and  J.  A.  Nelder.  Generalized  Linear  Models.  2"'*  ed.  London:  Chapman  & 

Hall,  1989. 

Myers,  Raymond  H.  Classical  and  Modem  Regression  with  Applications.  2"'*  ed.  Boston;  PWS- 
KENT  Publishing  Company,  1990. 

Nowlan,  F.  Stanley  and  Howard  F.  Heap.  Reliability  Centered  Maintenance.  Contract 
MDA  903-75-C-0349.  San  Francisco  CA:  United  Airlines,  29  December  1978 
(AD-A066579). 

O’Malley,  T.J.  and  Tovey  C.  Bachman.  Depot  Maintenance  in  the  Air  Force:  How 

Requirements  Are  Determined  and  How  They  Relate  to  Aircraft  Readiness  and 
Sustainability.  Contract  MDA903-85-C-0 139.  Bethesda  MA;  Logistics 
Management  Institute,  May  1990  (AD-A249  422). 

Presuitti,  Vic.  Briefing  Slides.  Readiness  Based  Sparing  (AAM).  HQAFMC/XPS. 

Wright  Patterson  AFB  OH,  23  March  1995. 

Ross,  Sheldon  M.  Introduction  to  Probability  Models.  5*  ed.  Boston:  Academic  Press,  Inc., 
1993. 


86 


Ross,  Sheldon  M.  Stochastic  Processes.  New  York;  John  Wiley  &  Sons  Inc.,  1983. 

Seevers,  Tim.  “Synergy;  An  Introduction  to  FAMMAS  in  Windows.”  Unpublished 
Report.  Synergy  Inc.,  Washington  DC,  1994. 

Siegel,  Joan,  ed.  Statistix.  Version  4.0,  Tallahassee  FI;  Analytical  Software,  1992. 

“Special  Report;  Managing  Nuclear  Maintenance,”  Electrical  World.  43-48  (February 
1990). 

Statistical  Sciences.  S-PLUS  Guide  to  Statistical  and  Mathematical  Analysis.  Version  3.2, 
Seattle;  StatSci,  a  division  of  MathSoft,  Inc.,  1993. 

Venables,  W.  N.,  and  B.  D.  Ripley.  Modem  Applied  Statistics  with  S-Plus.  New  York;  Springer- 
Verlag,  1994. 


87 


1 


VITA 

Capt.  Donald  F.  Hurry  was  bom  on  23  October  1967  in  Honolulu,  Hawaii.  He 
graduated  from  Plano  Senior  High  School,  Plano,  Texas,  in  1986  and  entered  the  United 
States  Air  Force  Academy.  In  December  of  1990,  he  graduated  with  a  Bachelor  of 
Science  degree  in  Operations  Research  and  received  his  commission  from  the  United 
States  Air  Force  Academy.  His  first  assignment  was  to  the  23rd  Fighter  Wing  at  Pope  Air 
Force  Base,  North  Carolina,  where  he  served  as  the  Officer  in  Charge  of  the  Aircraft 
Maintenance  Operational  Supply  Section  (AMOSS)  and  Fuels  Management  Flight 
Commander.  In  August  of  1994,  he  entered  the  School  of  Engineering,  Air  Force 
Institute  of  Technology.  After  graduation,  he  will  serve  as  a  member  of  the  Squadron 
Analysis  Section,  Air  Combat  Command  Staff,  Plans  Directorate,  Langley  AFB,  Virginia. 

Permanent  Address;  13827  Roswell  Dr. 

Carmel,  Indiana  46032 


88 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

0MB  No.  0704-0188 

Public  reDomnq  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instrurtions.  searching  existing  data  scwrcw. 
qathermq  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  as^rt  Of  this 
collection  of  information,  including  suggestions  for  reducing  this  burden,  to  Washington  Headauaners  Services.  Directorate  tor  ^ 

Davis  Highway.  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget.  Paperwork  Reduaion  Project  (0704-0 188).  Washington,  DC  20503. 

1.  AGENCY  USE  ONLY  (leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  OATES  COVERED 

March  1996  Master's  Thesis 

4.  TITLE  AND  SUBTITLE 

MEASURING  THE  IMPACT  OF  PROGRAMMED  DEPOT  MAINTENANCE 
FUNDING  SHORTFALLS  ON  WEAPON  SYSTEM  AVAILABILITY 

5.  FUNDING  NUMBERS 

6.  AUTHOR(S) 

Donald  F.  Hurry,  Captain  USAF 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Institute  of  Technology, 

WPAFB  OH  45433-7765 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

AFrr/GOR/ENS/96M-03 

9.  SPONSORING /MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

JDMAG 

Gentile  Station 

1080  Hamilton  Street 

Dayton,  OH  45444-5370 

10.  SPONSORING /MONITORING 

AGENCY  REPORT  NUMBER 

11.  SUPPLEMENTARY  NOTES 

12a.  DISTRIBUTION /AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

12b.  DISTRIBUTION  CODE 

13.  ABSTRACT  (Maximum  200  words) 

This  study  used  the  Poisson  regression  technique,  bootstrap  estimates,  and  the  Kaplan-Meier  estimate  for  survivor 
curves  to  determine  the  impact  of  Programmed  Depot  Maintenance  (PDM)  on  weapon  system  availability.  More 
specifically,  these  techniques  estimated  the  effect  in  the  number  failures  per  year  due  to  PDM,  but  the  bootstrap 
technique  ^so  estimated  the  effect  in  the  amount  of  downtime  experienced  1^  a  weapon  system  due  to  PDM.  Although 
the  Poisson  regression  model  did  not  pass  the  rigors  of  statistical  testing,  the  Poisson  regression  suggested  differences  in 
the  number  of  failures  per  year  between  PDM  and  no-PDM  weapon  systems.  The  results  of  the  bootstrap  estimates  for 
the  number  of  failures  per  year  and  amount  of  downtime  per  year  showed  that  the  no-PDM  weapon  systems  experienced 
approximately  1.3  failures  per  year  while  remaining  unserviceable  approximately  120  days  per  year,  and  PDM  weapon 
systems  experienced  approximately  .3  failures  per  year  while  remaining  unserviceable  approximately  30  days  per  year. 

PDM  reduced  the  number  of  failures  per  year  and  drastically  reduced  weapon  system  downtime  per  year.  PDM 
increased  weapon  system  availability  from  approximately  67  percent  to  approximately  92  percent. 

14.  SUBJECT  TERMS 

Programmed  Depot  Maintenance,  Poisson  Regression 

Bootstrap  Technique,  Vehicle  Reliability 

15.  NUMBER  OP  PAGES 

98 

16.  PRICE  CODE 

17.  SECURITY  CLASSIFICATION  18.  SECURITY  CLASSIFICATION  19.  SECURITY  CLASSIFICATION 

OF  REPORT  OF  THIS  PAGE  OF  ABSTRACT 

Unclassified  Unclassified  Unclassified 

20.  LIMITATION  OF  ABSTRACT 

UL 

NSN  7540-01-280*5500  Standard  Form  298  (Rev.  2-89) 


Prp^cnb^  bv  ANSI  Std 


